Sep 16 18:27:52.470223 ip-10-0-131-165 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:27:52.834772 ip-10-0-131-165 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:52.834772 ip-10-0-131-165 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:27:52.834772 ip-10-0-131-165 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:52.834772 ip-10-0-131-165 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:27:52.834772 ip-10-0-131-165 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:52.835671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.835598    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:27:52.838206 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838191    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.838206 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838206    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838210    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838212    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838215    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838218    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838220    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838227    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838230    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838233    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838235    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838238    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838241    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838243    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838246    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838248    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838251    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838253    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838258    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838262    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.838270 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838265    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838268    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838271    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838274    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838276    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838279    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838282    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838285    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838287    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838290    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838292    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838295    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838297    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838300    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838304    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838307    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838310    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838312    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838315    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.838716 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838317    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838321    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838323    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838326    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838328    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838331    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838333    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838336    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838338    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838341    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838343    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838346    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838348    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838351    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838354    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838357    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838359    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838362    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838365    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838367    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838370    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.839347 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838372    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838375    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838377    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838380    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838382    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838384    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838388    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838391    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838394    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838396    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838399    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838401    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838403    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838406    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838409    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838412    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838414    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838417    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838419    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838422    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.840071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838425    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838428    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838430    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838432    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838435    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838438    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838441    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838444    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838446    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838448    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838451    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838453    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838456    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838458    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838461    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838463    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838465    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838468    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.838470    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.841009 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840536    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840550    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840553    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840556    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840559    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840562    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840564    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840570    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840573    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840576    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840578    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840581    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840584    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840588    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840595    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840599    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840602    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840604    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840608    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.841572 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840611    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840613    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840616    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840619    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840621    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840632    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840635    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840641    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840644    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840647    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840649    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840652    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840654    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840657    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840660    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840663    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840666    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840669    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840675    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840677    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.842109 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840680    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840684    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840688    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840692    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840695    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840698    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840701    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840704    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840707    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840709    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840715    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840718    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840721    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840723    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840726    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840729    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840733    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840736    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840738    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.842594 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840741    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840744    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840746    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840751    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840754    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840756    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840759    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840761    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840764    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840767    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840769    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840772    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840774    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840777    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840779    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840782    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840787    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840790    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840793    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840795    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.843079 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840798    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840801    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840805    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840835    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.840998    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841004    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841007    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841010    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841013    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841016    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841019    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841023    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841026    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841029    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841031    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841034    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841037    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841040    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841043    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.843564 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841046    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841049    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841439    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841443    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841446    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841449    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841451    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841454    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841457    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841460    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841462    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841466    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841468    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841471    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841474    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841476    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841478    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841481    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841484    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841486    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.844043 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841489    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841491    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841494    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841496    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841499    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841503    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841507    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841510    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841513    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841515    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841519    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841521    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841524    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841527    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841529    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841532    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841535    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841537    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841540    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.844524 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841543    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841545    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841548    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841550    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841553    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841555    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841558    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841560    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841562    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841565    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841568    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841570    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841573    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841576    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841579    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841581    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841584    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841586    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841589    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.845010 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841591    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841594    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841596    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841598    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841601    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841603    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841606    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841608    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841612    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841615    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841619    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841621    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841624    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841628    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841631    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841633    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841636    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841639    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841642    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.845473 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841645    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841647    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841650    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841653    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841656    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841659    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841662    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841664    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841667    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841669    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841671    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841674    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841676    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841680    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841688    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841691    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841694    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841696    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841698    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841701    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.845923 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841704    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841706    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841708    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.841711    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841779    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841817    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841823    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841827    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841831    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841834    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841838    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841841    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841844    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841847    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841851    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841854    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841857    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841860    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841863    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841866    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841869    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841872    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841877    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841879    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:27:52.846495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841882    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841885    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841888    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841892    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841894    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841897    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841900    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841903    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841906    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841909    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841912    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841914    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841918    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841921    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841924    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841941    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841944    2576 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841947    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841951    2576 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841954    2576 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841957    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841960    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841963    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841972    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:27:52.847102 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841974    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841977    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841980    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841983    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841986    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841989    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841992    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841995    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.841998    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842000    2576 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842004    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842006    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842009    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842012    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842015    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842018    2576 flags.go:64] FLAG: --help="false"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842021    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-131-165.ec2.internal"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842023    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842026    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842029    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842032    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842035    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842038    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:27:52.847665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842041    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842043    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842046    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842049    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842052    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842055    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842057    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842060    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842063    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842066    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842069    2576 flags.go:64] FLAG: --lock-file=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842071    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842074    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842077    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842082    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842086    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842088    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842091    2576 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842094    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842097    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842099    2576 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842102    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842107    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842110    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842114    2576 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:27:52.848260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842117    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842120    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842122    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842125    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842128    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842131    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842134    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842140    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842143    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842146    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842149    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842152    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842157    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842160    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842163    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842166    2576 flags.go:64] FLAG: --port="10250"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842171    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842174    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d38a9315616efc8"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842177    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842180    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842183    2576 flags.go:64] FLAG: --register-node="true"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842185    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842190    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842194    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:27:52.848886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842196    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842199    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842203    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842205    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842208    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842211    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842214    2576 flags.go:64] FLAG: --runonce="false"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842217    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842220    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842222    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842225    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842228    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842231    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842234    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842236    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842239    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842242    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842245    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842247    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842250    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842253    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842255    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842260    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842263    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842267    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:27:52.849497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842271    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842274    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842277    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842279    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842282    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842286    2576 flags.go:64] FLAG: --v="2"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842290    2576 flags.go:64] FLAG: --version="false"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842294    2576 flags.go:64] FLAG: --vmodule=""
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842298    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.842301    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842451    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842455    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842458    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842461    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842464    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842467    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842469    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842472    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842474    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842477    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842479    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842482    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.850219 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842484    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842487    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842489    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842492    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842495    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842497    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842500    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842502    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842505    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842509    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842512    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842515    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842517    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842519    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842522    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842525    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842528    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842530    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842533    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842536    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.850753 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842538    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842541    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842543    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842546    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842548    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842551    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842553    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842556    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842559    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842563    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842567    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842571    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842575    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842578    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842580    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842583    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842585    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842588    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842590    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842593    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.851251 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842595    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842600    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842602    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842605    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842607    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842610    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842612    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842616    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842620    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842625    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842629    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842633    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842636    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842639    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842641    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842644    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842647    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842650    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842653    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.851745 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842655    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842658    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842661    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842664    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842666    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842668    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842671    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842674    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842676    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842679    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842681    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842684    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842686    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842689    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842692    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842694    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842697    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842699    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842702    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842704    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.852231 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842709    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842711    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842714    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842716    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842719    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842735    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842738    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.842742    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.843407    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.849878    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.849891    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.849910    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.849919    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.849986    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.852731 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.849992    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.849995    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.849998    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850001    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850012    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850016    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850019    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850022    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850025    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850027    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850030    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850033    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850035    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850038    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850041    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850043    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850046    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850048    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850051    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.853092 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850053    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850056    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850058    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850060    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850063    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850065    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850068    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850070    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850072    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850077    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850080    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850083    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850085    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850087    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850090    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850092    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850095    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850097    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850100    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.853579 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850102    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850104    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850107    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850110    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850112    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850115    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850117    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850120    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850122    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850125    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850127    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850129    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850132    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850134    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850137    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850140    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850142    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850145    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850147    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850149    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.854046 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850152    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850155    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850158    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850161    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850163    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850166    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850168    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850170    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850173    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850176    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850179    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850181    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850183    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850186    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850188    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850191    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850193    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850196    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850199    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850201    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.854525 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850204    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850208    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850211    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850214    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850216    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850219    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850221    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850224    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850227    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850229    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850232    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850236    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850239    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850242    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850245    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850248    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850251    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850254    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.855061 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850257    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850261    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.850266    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850371    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850376    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850378    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850381    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850384    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850387    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850390    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850392    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850395    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850398    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850401    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:52.855484 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850403    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850406    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850408    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850411    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850413    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850416    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850418    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850421    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850423    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850426    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850428    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850431    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850433    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850436    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850439    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850441    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850444    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850450    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850454    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:52.855818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850457    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850460    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850462    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850465    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850467    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850470    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850472    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850474    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850477    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850479    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850482    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850485    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850487    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850490    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850492    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850495    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850497    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850500    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850502    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850505    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:52.856282 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850507    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850510    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850512    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850514    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850517    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850520    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850522    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850525    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850527    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850530    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850532    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850535    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850538    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850542    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850545    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850548    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850551    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850554    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850556    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850559    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:52.856762 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850562    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850565    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850568    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850570    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850573    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850575    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850578    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850580    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850583    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850585    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850587    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850590    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850593    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850595    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850597    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850600    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850602    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850605    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850607    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850611    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:52.857253 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850614    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850616    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850619    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850621    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850624    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850628    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850631    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850634    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:52.850637    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.850642    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.850729    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.853417    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.854408    2576 server.go:1019] "Starting client certificate rotation"
Sep 16 18:27:52.857736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.854505    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:27:52.858071 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.854554    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:27:52.858071 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.857051    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:27:52.858123 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.858086    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:27:52.870841 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.870827    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:27:52.874690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.874675    2576 log.go:25] "Validated CRI v1 image API"
Sep 16 18:27:52.875583 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.875568    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:27:52.881081 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.881061    2576 fs.go:135] Filesystem UUIDs: map[34b40bd8-7812-4a93-8e7e-e8f224c3af72:/dev/nvme0n1p3 5265e669-d24b-4bf2-acf7-651c616c351b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 16 18:27:52.881154 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.881079    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:27:52.886314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.886201    2576 manager.go:216] Machine: {Timestamp:2026-09-16 18:27:52.88481795 +0000 UTC m=+0.305304802 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099972 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2db979d25545bbcd28bd18bdc3d6f5 SystemUUID:ec2db979-d255-45bb-cd28-bd18bdc3d6f5 BootID:a48746bd-6324-4b4a-877e-3e0c9ece5908 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dc:84:d9:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dc:84:d9:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:a7:98:2b:c9:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:27:52.886314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.886306    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:27:52.886434 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.886376    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:27:52.886574 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.886553    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:27:52.889738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.889716    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:27:52.889896 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.889741    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:27:52.890460 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.890451    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:27:52.890498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.890463    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:27:52.890498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.890471    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:27:52.890998 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.890988    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:27:52.891642 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.891633    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:27:52.894048 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894037    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:27:52.894156 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894147    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:27:52.894186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894175    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:27:52.894186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894185    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:27:52.894278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894202    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:27:52.894278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894211    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:27:52.894278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894219    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:27:52.894552 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894540    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:27:52.894609 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.894558    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:27:52.896353 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.896337    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:27:52.902119 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.902104    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:27:52.902185 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.902135    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:27:52.903717 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903706    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903722    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903728    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903733    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903741    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903746    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:27:52.903760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903759    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:27:52.903909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903766    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:27:52.903909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903773    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:27:52.903909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903778    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:27:52.903909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903786    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:27:52.903909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.903791    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:27:52.905001 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.904990    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:27:52.905040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.905003    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:27:52.906704 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.906665    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:27:52.906751 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.906681    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:27:52.908662 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.908650    2576 server.go:1268] "Started kubelet"
Sep 16 18:27:52.908771 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.908746    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:27:52.908917 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.908744    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:27:52.909099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.909079    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:27:52.909599 ip-10-0-131-165 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:27:52.910721 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.910501    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:27:52.913279 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.913247    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xvwht"
Sep 16 18:27:52.914382 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.914357    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:27:52.914522 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.914499    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:27:52.915971 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.915949    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:27:52.917107 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917092    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:27:52.917304 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.917276    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:52.917602 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917585    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:27:52.917676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917605    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:27:52.917676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917591    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:27:52.917772 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917690    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:27:52.917772 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917697    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:27:52.917983 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917959    2576 factory.go:55] Registering systemd factory
Sep 16 18:27:52.918068 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.917987    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:27:52.918209 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918186    2576 factory.go:153] Registering CRI-O factory
Sep 16 18:27:52.918251 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918211    2576 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:27:52.918280 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918262    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:27:52.918309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918285    2576 factory.go:103] Registering Raw factory
Sep 16 18:27:52.918309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918303    2576 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:27:52.918720 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.918700    2576 manager.go:349] Starting recovery of all containers
Sep 16 18:27:52.919026 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.919009    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xvwht"
Sep 16 18:27:52.920684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.920664    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:27:52.920920 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.920894    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:27:52.921193 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.921170    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:27:52.922408 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.921118    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-165.ec2.internal.18d5e0814db2e8d0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-165.ec2.internal,UID:ip-10-0-131-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-165.ec2.internal,},FirstTimestamp:2026-09-16 18:27:52.90861384 +0000 UTC m=+0.329100691,LastTimestamp:2026-09-16 18:27:52.90861384 +0000 UTC m=+0.329100691,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-165.ec2.internal,}"
Sep 16 18:27:52.925650 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.925614    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:27:52.930538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.930520    2576 manager.go:354] Recovery completed
Sep 16 18:27:52.932580 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.932557    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 16 18:27:52.935482 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.935468    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:52.937467 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.937449    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:52.937537 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.937481    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:52.937537 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.937491    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:52.938376 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.938356    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:52.938479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.938424    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:27:52.938479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.938447    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:27:52.938479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.938472    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:27:52.940576 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940557    2576 policy_none.go:49] "None policy: Start"
Sep 16 18:27:52.940576 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940577    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:27:52.940719 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940592    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:27:52.940851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940835    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:52.940908 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940863    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:52.940908 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.940889    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:52.941663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.941652    2576 policy_none.go:47] "Start"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.978470    2576 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.978499    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.978508    2576 server.go:87] "Starting device plugin registration server"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.978791    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.978803    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.978950    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.979036    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:52.979045    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.979416    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:27:52.980793 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:52.979457    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.004002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.003978    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:27:53.004076 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.004034    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:27:53.004076 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.004053    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:27:53.004148 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.004087    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:27:53.006269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.006252    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:53.079272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.079254    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.080055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.080041    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.080137 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.080071    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.080137 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.080086    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.080137 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.080112    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.088017 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.087958    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.088017 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.087983    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-165.ec2.internal\": node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.101298 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.101285    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.104349 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.104333    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"]
Sep 16 18:27:53.104405 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.104388    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.105896 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.105875    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.106007 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.105908    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.106007 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.105946    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.107165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107149    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.107244 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.107304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107253    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.107961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107944    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.108058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107971    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.108058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107951    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.108058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.107986    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.108058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.108011    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.108058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.108027    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.109332 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.109312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.109421 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.109343    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.110011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.109984    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.110011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.110012    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.110140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.110023    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.137450 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.137430    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-165.ec2.internal\" not found" node="ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.141630 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.141610    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-165.ec2.internal\" not found" node="ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.202299 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.202277    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.218782 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.218759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.218869 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.218789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.218869 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.218813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16cc17fcb523a0845ceaeb734d781a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"16cc17fcb523a0845ceaeb734d781a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.303160 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.303139    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.319399 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.319455 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.319490 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16cc17fcb523a0845ceaeb734d781a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"16cc17fcb523a0845ceaeb734d781a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.319490 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16cc17fcb523a0845ceaeb734d781a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"16cc17fcb523a0845ceaeb734d781a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.319553 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319481    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.319553 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.319506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e544008ed7b87edba555758daa6cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"03e544008ed7b87edba555758daa6cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.403821 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.403771    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.439259 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.439238    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.444901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.444883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:53.504482 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.504451    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.605011 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.604983    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.705449 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.705427    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.806059 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.806036    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.854497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.854469    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:27:53.854881 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.854607    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:27:53.906900 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:53.906881    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:53.922874 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.922842    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:22:53 +0000 UTC" deadline="2028-02-14 06:14:19.703523165 +0000 UTC"
Sep 16 18:27:53.922961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.922880    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12371h46m25.780656709s"
Sep 16 18:27:53.931762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.931741    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:53.937423 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:53.937402    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:54.007034 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.007011    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:54.096744 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:54.096703    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cc17fcb523a0845ceaeb734d781a6a.slice/crio-c843e9bd03265350b64366badd2af5d1efbff31f55a261547328a3e51405258a WatchSource:0}: Error finding container c843e9bd03265350b64366badd2af5d1efbff31f55a261547328a3e51405258a: Status 404 returned error can't find the container with id c843e9bd03265350b64366badd2af5d1efbff31f55a261547328a3e51405258a
Sep 16 18:27:54.097060 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:54.097034    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e544008ed7b87edba555758daa6cf5.slice/crio-6bb704d4ba330a9153793d8c812333b1048bae8c06306975ce371038ac026726 WatchSource:0}: Error finding container 6bb704d4ba330a9153793d8c812333b1048bae8c06306975ce371038ac026726: Status 404 returned error can't find the container with id 6bb704d4ba330a9153793d8c812333b1048bae8c06306975ce371038ac026726
Sep 16 18:27:54.100737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.100701    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:27:54.108044 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.108028    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:54.208409 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.208387    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:54.308962 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.308905    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:54.409647 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.409627    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Sep 16 18:27:54.420519 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.420500    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:54.517362 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.517333    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:54.535840 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.535817    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:27:54.536763 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.536749    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Sep 16 18:27:54.545584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.545570    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:27:54.895272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.895238    2576 apiserver.go:52] "Watching apiserver"
Sep 16 18:27:54.903141 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.903113    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:27:54.903580 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.903543    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-q5qr4","openshift-multus/multus-additional-cni-plugins-m4bmk","openshift-multus/network-metrics-daemon-m49ct","openshift-network-diagnostics/network-check-target-fcbbv","openshift-network-operator/iptables-alerter-bjz6z","kube-system/konnectivity-agent-x8zxq","openshift-dns/node-resolver-xxksn","openshift-image-registry/node-ca-5g7ts","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal","openshift-multus/multus-lg8bf","openshift-ovn-kubernetes/ovnkube-node-gm67r","kube-system/global-pull-secret-syncer-5fns9","kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"]
Sep 16 18:27:54.906285 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.906257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:54.909287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.908596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.909287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.908879    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdm9f\""
Sep 16 18:27:54.909287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.909134    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.909287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.909177    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.911560 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.911423    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.911673 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.911597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.911957 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.911937    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:27:54.912684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.912425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:54.912684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.912488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:27:54.912684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.912507    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zqg8m\""
Sep 16 18:27:54.912684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.912636    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:27:54.912684 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.912497    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:27:54.912957 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.912735    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:27:54.914953 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.914798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:54.914953 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.914869    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:27:54.917197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.917175    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:27:54.917936 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.917913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:54.919952 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.919920    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:54.921270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.921244    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.921368 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.921279    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9hjz7\""
Sep 16 18:27:54.921526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.921508    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.921645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.921627    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:27:54.922158 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.922116    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:27:54.922476 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.922458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:27:54.922564 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.922492    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4wtnx\""
Sep 16 18:27:54.922765 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.922745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:54.925611 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.925588    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.925925 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.925788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hwn5q\""
Sep 16 18:27:54.925925 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.925805    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.925925 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.925794    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:27:54.926428 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.926331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.928853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-systemd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.928883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.928904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.928906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4fvb\" (UniqueName: \"kubernetes.io/projected/f937f5ba-2468-4b6e-8037-e2a4107f754c-kube-api-access-n4fvb\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/182dea7c-2698-468e-88bc-428ad9033d36-agent-certs\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/182dea7c-2698-468e-88bc-428ad9033d36-konnectivity-ca\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz69q\" (UniqueName: \"kubernetes.io/projected/2fb4ca0a-da96-4546-95d9-59121e571d61-kube-api-access-fz69q\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fpnj\" (UniqueName: \"kubernetes.io/projected/80d732e2-c418-40e7-bd17-6258bf894253-kube-api-access-7fpnj\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-var-lib-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-bin\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-netd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-kubelet\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fb4ca0a-da96-4546-95d9-59121e571d61-ovn-node-metrics-cert\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f937f5ba-2468-4b6e-8037-e2a4107f754c-iptables-alerter-script\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:54.929731 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b06ca00-79eb-4e58-90bb-af7533456332-hosts-file\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-node-log\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-script-lib\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f937f5ba-2468-4b6e-8037-e2a4107f754c-host-slash\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-log-socket\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-netns\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-ovn\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-env-overrides\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.929974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-slash\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b06ca00-79eb-4e58-90bb-af7533456332-tmp-dir\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shqc9\" (UniqueName: \"kubernetes.io/projected/0b06ca00-79eb-4e58-90bb-af7533456332-kube-api-access-shqc9\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-systemd-units\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-etc-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-config\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930312    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930332    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.930664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.930442    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2w6gn\""
Sep 16 18:27:54.931640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.931283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:54.932742 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.932508    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:27:54.933287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933030    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.933287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933091    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-svphk\""
Sep 16 18:27:54.933287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933156    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.933546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933527    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:27:54.933605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933571    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:27:54.933695 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933670    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w6hmp\""
Sep 16 18:27:54.933789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933720    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:27:54.933789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933768    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:27:54.933789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.933768    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.934277 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.934176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:54.936202 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.936178    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.936495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.936474    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zpt92\""
Sep 16 18:27:54.936572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.936555    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.936950 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.936927    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:54.937074 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:54.936994    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:27:54.951086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.951063    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vf6hs"
Sep 16 18:27:54.959047 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:54.959027    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vf6hs"
Sep 16 18:27:55.009799 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.009758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"03e544008ed7b87edba555758daa6cf5","Type":"ContainerStarted","Data":"6bb704d4ba330a9153793d8c812333b1048bae8c06306975ce371038ac026726"}
Sep 16 18:27:55.011247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.011221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" event={"ID":"16cc17fcb523a0845ceaeb734d781a6a","Type":"ContainerStarted","Data":"c843e9bd03265350b64366badd2af5d1efbff31f55a261547328a3e51405258a"}
Sep 16 18:27:55.019407 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.019382    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:27:55.030373 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-device-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.030488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-socket-dir-parent\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.030488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-kubernetes\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.030488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-systemd\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.030488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fpnj\" (UniqueName: \"kubernetes.io/projected/80d732e2-c418-40e7-bd17-6258bf894253-kube-api-access-7fpnj\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:55.030651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-var-lib-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-var-lib-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5brwn\" (UniqueName: \"kubernetes.io/projected/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kube-api-access-5brwn\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.030801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.030801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.030801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-tuned\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.030926 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-kubelet\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030926 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030926 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-kubelet\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.030926 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f937f5ba-2468-4b6e-8037-e2a4107f754c-iptables-alerter-script\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-netns\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-bin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.030917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-conf-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.031121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-node-log\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-node-log\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-modprobe-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b06ca00-79eb-4e58-90bb-af7533456332-tmp-dir\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-host\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-script-lib\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f937f5ba-2468-4b6e-8037-e2a4107f754c-host-slash\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-log-socket\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-slash\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-serviceca\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.031468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-log-socket\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f937f5ba-2468-4b6e-8037-e2a4107f754c-iptables-alerter-script\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-slash\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f937f5ba-2468-4b6e-8037-e2a4107f754c-host-slash\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrgv\" (UniqueName: \"kubernetes.io/projected/4c5b3366-d2a1-4cd2-90d1-c207b8412337-kube-api-access-mgrgv\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b06ca00-79eb-4e58-90bb-af7533456332-tmp-dir\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-env-overrides\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-k8s-cni-cncf-io\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtb29\" (UniqueName: \"kubernetes.io/projected/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-kube-api-access-rtb29\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.032118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-var-lib-kubelet\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-script-lib\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032145    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-systemd-units\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-env-overrides\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.031992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-systemd-units\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-etc-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-systemd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032526    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032541    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-socket-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-system-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysconfig\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.032601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-lib-modules\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz69q\" (UniqueName: \"kubernetes.io/projected/2fb4ca0a-da96-4546-95d9-59121e571d61-kube-api-access-fz69q\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cni-binary-copy\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch45b\" (UniqueName: \"kubernetes.io/projected/03db53a4-ede7-4baf-825b-8757e23d1a01-kube-api-access-ch45b\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-bin\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-netd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032749    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-registration-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-multus\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-daemon-config\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-etc-kubernetes\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.032859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-dbus\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033045    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fb4ca0a-da96-4546-95d9-59121e571d61-ovn-node-metrics-cert\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b06ca00-79eb-4e58-90bb-af7533456332-hosts-file\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzwtz\" (UniqueName: \"kubernetes.io/projected/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-kube-api-access-nzwtz\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-kubelet\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033155 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033154    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-host\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cnibin\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-sys\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shqc9\" (UniqueName: \"kubernetes.io/projected/0b06ca00-79eb-4e58-90bb-af7533456332-kube-api-access-shqc9\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-netns\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-ovn\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-hostroot\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-conf\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-run\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-kubelet-config\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-sys-fs\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cnibin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-multus-certs\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-tmp\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-config\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-system-cni-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.033621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4fvb\" (UniqueName: \"kubernetes.io/projected/f937f5ba-2468-4b6e-8037-e2a4107f754c-kube-api-access-n4fvb\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.034146 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/182dea7c-2698-468e-88bc-428ad9033d36-agent-certs\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:55.034146 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.033675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/182dea7c-2698-468e-88bc-428ad9033d36-konnectivity-ca\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:55.034305 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-etc-openvswitch\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.034305 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/182dea7c-2698-468e-88bc-428ad9033d36-konnectivity-ca\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:55.034425 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-systemd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.034546 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.034527    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.034647 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.034619    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.534576265 +0000 UTC m=+2.955063129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.034912 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-bin\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.035029 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-cni-netd\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.035029 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-host-run-netns\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.035029 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.034991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fb4ca0a-da96-4546-95d9-59121e571d61-run-ovn\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.035276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.035222    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:27:55.035334 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.035278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b06ca00-79eb-4e58-90bb-af7533456332-hosts-file\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.035526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.035507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fb4ca0a-da96-4546-95d9-59121e571d61-ovnkube-config\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.040565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.040434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fb4ca0a-da96-4546-95d9-59121e571d61-ovn-node-metrics-cert\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.040660 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.040586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/182dea7c-2698-468e-88bc-428ad9033d36-agent-certs\") pod \"konnectivity-agent-x8zxq\" (UID: \"182dea7c-2698-468e-88bc-428ad9033d36\") " pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:55.041234 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.041213    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fpnj\" (UniqueName: \"kubernetes.io/projected/80d732e2-c418-40e7-bd17-6258bf894253-kube-api-access-7fpnj\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:55.043231 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.043197    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:55.043327 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.043241    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:55.045225 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.045038    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.045225 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.045136    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.545118251 +0000 UTC m=+2.965605115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.049084 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.049062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4fvb\" (UniqueName: \"kubernetes.io/projected/f937f5ba-2468-4b6e-8037-e2a4107f754c-kube-api-access-n4fvb\") pod \"iptables-alerter-bjz6z\" (UID: \"f937f5ba-2468-4b6e-8037-e2a4107f754c\") " pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.051744 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.051720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shqc9\" (UniqueName: \"kubernetes.io/projected/0b06ca00-79eb-4e58-90bb-af7533456332-kube-api-access-shqc9\") pod \"node-resolver-xxksn\" (UID: \"0b06ca00-79eb-4e58-90bb-af7533456332\") " pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.052941 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.052916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz69q\" (UniqueName: \"kubernetes.io/projected/2fb4ca0a-da96-4546-95d9-59121e571d61-kube-api-access-fz69q\") pod \"ovnkube-node-gm67r\" (UID: \"2fb4ca0a-da96-4546-95d9-59121e571d61\") " pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.134135 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-hostroot\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134135 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-conf\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-run\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-kubelet-config\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-hostroot\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-sys-fs\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-sys-fs\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cnibin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-multus-certs\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-tmp\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-system-cni-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-conf\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-device-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-socket-dir-parent\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-kubernetes\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-systemd\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5brwn\" (UniqueName: \"kubernetes.io/projected/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kube-api-access-5brwn\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-run\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-tuned\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-kubelet-config\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-systemd\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-kubernetes\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.134773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cnibin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-multus-certs\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysctl-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.134924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-device-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135123    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-socket-dir-parent\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-netns\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-system-cni-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-bin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-bin\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-conf-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-netns\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-modprobe-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-host\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.135691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-serviceca\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrgv\" (UniqueName: \"kubernetes.io/projected/4c5b3366-d2a1-4cd2-90d1-c207b8412337-kube-api-access-mgrgv\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-k8s-cni-cncf-io\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtb29\" (UniqueName: \"kubernetes.io/projected/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-kube-api-access-rtb29\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-var-lib-kubelet\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-host\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135755    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-modprobe-d\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-socket-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-serviceca\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-system-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysconfig\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.136538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-lib-modules\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cni-binary-copy\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch45b\" (UniqueName: \"kubernetes.io/projected/03db53a4-ede7-4baf-825b-8757e23d1a01-kube-api-access-ch45b\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-registration-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-multus\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-daemon-config\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-etc-kubernetes\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-lib-modules\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-dbus\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-sysconfig\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.135257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-conf-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-system-cni-dir\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzwtz\" (UniqueName: \"kubernetes.io/projected/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-kube-api-access-nzwtz\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-kubelet\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-host\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cnibin\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-sys\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.137395 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.136337    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-run-k8s-cni-cncf-io\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c5b3366-d2a1-4cd2-90d1-c207b8412337-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.136403    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.636386516 +0000 UTC m=+3.056873375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-socket-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-kubelet\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-host\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c049995-694f-4ace-b1a7-6ea9f6d047a7-dbus\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c5b3366-d2a1-4cd2-90d1-c207b8412337-cnibin\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ab9fdd5-b061-4357-ae1f-add31fb423ff-registration-dir\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-sys\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-multus-daemon-config\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-etc-kubernetes\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03db53a4-ede7-4baf-825b-8757e23d1a01-var-lib-kubelet\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.136777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-host-var-lib-cni-multus\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.138300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.137080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-cni-binary-copy\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.139587 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.139560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-tmp\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.141427 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.141391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03db53a4-ede7-4baf-825b-8757e23d1a01-etc-tuned\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.150167 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.150143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtb29\" (UniqueName: \"kubernetes.io/projected/c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19-kube-api-access-rtb29\") pod \"multus-lg8bf\" (UID: \"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19\") " pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.151376 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.151331    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5brwn\" (UniqueName: \"kubernetes.io/projected/3ab9fdd5-b061-4357-ae1f-add31fb423ff-kube-api-access-5brwn\") pod \"aws-ebs-csi-driver-node-9gt8x\" (UID: \"3ab9fdd5-b061-4357-ae1f-add31fb423ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.151475 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.151460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzwtz\" (UniqueName: \"kubernetes.io/projected/8eed79a5-6b78-4e0d-8516-cdea9b3f726d-kube-api-access-nzwtz\") pod \"node-ca-5g7ts\" (UID: \"8eed79a5-6b78-4e0d-8516-cdea9b3f726d\") " pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.152708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.152367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch45b\" (UniqueName: \"kubernetes.io/projected/03db53a4-ede7-4baf-825b-8757e23d1a01-kube-api-access-ch45b\") pod \"tuned-q5qr4\" (UID: \"03db53a4-ede7-4baf-825b-8757e23d1a01\") " pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.152949 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.152872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrgv\" (UniqueName: \"kubernetes.io/projected/4c5b3366-d2a1-4cd2-90d1-c207b8412337-kube-api-access-mgrgv\") pod \"multus-additional-cni-plugins-m4bmk\" (UID: \"4c5b3366-d2a1-4cd2-90d1-c207b8412337\") " pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.220882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.220803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xxksn"
Sep 16 18:27:55.229993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.229965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:27:55.241149 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.241128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjz6z"
Sep 16 18:27:55.249077 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.249057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:27:55.258127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.258105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x"
Sep 16 18:27:55.267973 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.267951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5g7ts"
Sep 16 18:27:55.274840 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.274814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m4bmk"
Sep 16 18:27:55.283988 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.283961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lg8bf"
Sep 16 18:27:55.286926 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.286908    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:55.288738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.288720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q5qr4"
Sep 16 18:27:55.539998 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.539898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:55.540147 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.540072    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.540147 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.540144    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.540128383 +0000 UTC m=+3.960615222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.641247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.641220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:55.641436 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.641272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:55.641436 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641404    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.641436 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641420    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:55.641436 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641438    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:55.641658 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641451    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.641658 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641469    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.641449913 +0000 UTC m=+4.061936754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.641658 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:55.641497    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.641482513 +0000 UTC m=+4.061969369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.962485 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.962446    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:22:55 +0000 UTC" deadline="2028-06-25 22:54:59.444100178 +0000 UTC"
Sep 16 18:27:55.962485 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:55.962476    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15556h27m3.481627904s"
Sep 16 18:27:56.216840 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:56.216812    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182dea7c_2698_468e_88bc_428ad9033d36.slice/crio-838f95585e479382952faadade951bea23889c0838b50035c7ab8844014825af WatchSource:0}: Error finding container 838f95585e479382952faadade951bea23889c0838b50035c7ab8844014825af: Status 404 returned error can't find the container with id 838f95585e479382952faadade951bea23889c0838b50035c7ab8844014825af
Sep 16 18:27:56.220058 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:56.220011    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb4ca0a_da96_4546_95d9_59121e571d61.slice/crio-378b47e5a8d1b0571e3c439f513f170878569af22d4e934041c38c0be94b9636 WatchSource:0}: Error finding container 378b47e5a8d1b0571e3c439f513f170878569af22d4e934041c38c0be94b9636: Status 404 returned error can't find the container with id 378b47e5a8d1b0571e3c439f513f170878569af22d4e934041c38c0be94b9636
Sep 16 18:27:56.223123 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:56.222626    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eed79a5_6b78_4e0d_8516_cdea9b3f726d.slice/crio-bb8bb9f51a938a931f779227c7b2b1c013ec4586a3770b8a108214746b999a51 WatchSource:0}: Error finding container bb8bb9f51a938a931f779227c7b2b1c013ec4586a3770b8a108214746b999a51: Status 404 returned error can't find the container with id bb8bb9f51a938a931f779227c7b2b1c013ec4586a3770b8a108214746b999a51
Sep 16 18:27:56.223974 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:56.223605    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5b3366_d2a1_4cd2_90d1_c207b8412337.slice/crio-81a3bfc6036a23b1b70c04147d86ff3f244acb6fb9e2d0a111e3482d8d568c0d WatchSource:0}: Error finding container 81a3bfc6036a23b1b70c04147d86ff3f244acb6fb9e2d0a111e3482d8d568c0d: Status 404 returned error can't find the container with id 81a3bfc6036a23b1b70c04147d86ff3f244acb6fb9e2d0a111e3482d8d568c0d
Sep 16 18:27:56.223974 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:27:56.223886    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab9fdd5_b061_4357_ae1f_add31fb423ff.slice/crio-7dc033ece0405ec91ce974db438ee38b24ab4dd31331287e87f6befec3e86a41 WatchSource:0}: Error finding container 7dc033ece0405ec91ce974db438ee38b24ab4dd31331287e87f6befec3e86a41: Status 404 returned error can't find the container with id 7dc033ece0405ec91ce974db438ee38b24ab4dd31331287e87f6befec3e86a41
Sep 16 18:27:56.548170 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:56.548140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:56.548272 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.548244    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:56.548310 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.548294    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.548279358 +0000 UTC m=+5.968766198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:56.648614 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:56.648580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:56.648753 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:56.648643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:56.648753 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648701    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:56.648874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648770    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.648750208 +0000 UTC m=+6.069237053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:56.648874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648785    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:56.648874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648802    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:56.648874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648813    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:56.648874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:56.648850    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.648838922 +0000 UTC m=+6.069325761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:56.963640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:56.963395    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:22:55 +0000 UTC" deadline="2028-06-15 07:33:41.690329542 +0000 UTC"
Sep 16 18:27:56.963640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:56.963454    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15301h5m44.726879814s"
Sep 16 18:27:57.006865 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.006828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:57.007024 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:57.006969    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:27:57.007409 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.007386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:57.007511 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:57.007485    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:27:57.007571 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.007554    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:57.007684 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:57.007642    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:27:57.039116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.039081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" event={"ID":"16cc17fcb523a0845ceaeb734d781a6a","Type":"ContainerStarted","Data":"980fe5d47bd517314478d3c6b3702e792c94b215990f576e1653ee8506027651"}
Sep 16 18:27:57.049612 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.049581    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xxksn" event={"ID":"0b06ca00-79eb-4e58-90bb-af7533456332","Type":"ContainerStarted","Data":"af6dc03f092ebc340c55d9c4fea1832d1ea73c44786590bccb97ca51c2e06a66"}
Sep 16 18:27:57.056371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.056307    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" podStartSLOduration=3.056287929 podStartE2EDuration="3.056287929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:57.055660041 +0000 UTC m=+4.476146909" watchObservedRunningTime="2026-09-16 18:27:57.056287929 +0000 UTC m=+4.476774793"
Sep 16 18:27:57.075540 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.075509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lg8bf" event={"ID":"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19","Type":"ContainerStarted","Data":"d23e32333c22b7673ede3633e89b3184b2c5fa72304d254400f5a3fad18ae29e"}
Sep 16 18:27:57.093272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.091863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q5qr4" event={"ID":"03db53a4-ede7-4baf-825b-8757e23d1a01","Type":"ContainerStarted","Data":"6c8199b7888913f7f0021ea194083a668dc4e4d5f03215a1b66bdb9a4798b9ad"}
Sep 16 18:27:57.104696 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.104617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjz6z" event={"ID":"f937f5ba-2468-4b6e-8037-e2a4107f754c","Type":"ContainerStarted","Data":"2818ac9bcb8fb0874fe1951adcdd1e7222a95ee7b37cd4bf71aedcd6e28d6fd8"}
Sep 16 18:27:57.117121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.116999    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x" event={"ID":"3ab9fdd5-b061-4357-ae1f-add31fb423ff","Type":"ContainerStarted","Data":"7dc033ece0405ec91ce974db438ee38b24ab4dd31331287e87f6befec3e86a41"}
Sep 16 18:27:57.125918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.125847    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5g7ts" event={"ID":"8eed79a5-6b78-4e0d-8516-cdea9b3f726d","Type":"ContainerStarted","Data":"bb8bb9f51a938a931f779227c7b2b1c013ec4586a3770b8a108214746b999a51"}
Sep 16 18:27:57.131533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.131461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerStarted","Data":"81a3bfc6036a23b1b70c04147d86ff3f244acb6fb9e2d0a111e3482d8d568c0d"}
Sep 16 18:27:57.141397 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.141299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"378b47e5a8d1b0571e3c439f513f170878569af22d4e934041c38c0be94b9636"}
Sep 16 18:27:57.144921 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.144854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x8zxq" event={"ID":"182dea7c-2698-468e-88bc-428ad9033d36","Type":"ContainerStarted","Data":"838f95585e479382952faadade951bea23889c0838b50035c7ab8844014825af"}
Sep 16 18:27:57.148777 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.147985    2576 generic.go:356] "Generic (PLEG): container finished" podID="03e544008ed7b87edba555758daa6cf5" containerID="2a5ed4c0978f77343ee8454aff340265dfe97e7c3504c6baa36f21d9ea2b3dc0" exitCode=0
Sep 16 18:27:57.148777 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:57.148037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"03e544008ed7b87edba555758daa6cf5","Type":"ContainerDied","Data":"2a5ed4c0978f77343ee8454aff340265dfe97e7c3504c6baa36f21d9ea2b3dc0"}
Sep 16 18:27:58.169422 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:58.169386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"03e544008ed7b87edba555758daa6cf5","Type":"ContainerStarted","Data":"52fd3a6d94e2a20d0f12b6901981688a54309b1326425fe502d1ad03d41ad9a0"}
Sep 16 18:27:58.569091 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:58.568733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:58.569268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.569174    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:58.569268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.569239    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.569220007 +0000 UTC m=+9.989706847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:58.670379 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:58.670343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:58.670546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:58.670415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:58.670638 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.670553    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:58.670638 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.670576    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:58.670638 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.670589    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:58.670816 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.670658    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.670628502 +0000 UTC m=+10.091115348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:58.671105 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.671065    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:58.671195 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:58.671119    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.671102581 +0000 UTC m=+10.091589427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:59.006148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:59.006114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:27:59.006339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:59.006128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:27:59.006339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:27:59.006273    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:27:59.006339 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:59.006269    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:27:59.006515 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:59.006373    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:27:59.006515 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:27:59.006457    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:01.007414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:01.007561    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:01.007991    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:01.008088    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:01.008151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:01.008278 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:01.008231    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:02.605402 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:02.605362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:02.605838 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.605479    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:02.605838 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.605529    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.605514904 +0000 UTC m=+18.026001744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:02.705818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:02.705903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.705981    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.706003    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.706014    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.706018    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.706062    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.706049661 +0000 UTC m=+18.126536498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:02.706072 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:02.706077    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.706070763 +0000 UTC m=+18.126557602 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:03.005895 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:03.005864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:03.006070 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:03.005991    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:03.006382 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:03.006357    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:03.006478 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:03.006459    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:03.006589 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:03.006578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:03.006687 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:03.006662    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:05.007000 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:05.006967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:05.007450 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:05.007076    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:05.007509 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:05.007467    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:05.007580 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:05.007563    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:05.007636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:05.007609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:05.007692 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:05.007671    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:07.005204 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:07.005127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:07.005204 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:07.005159    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:07.005593 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:07.005240    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:07.005593 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:07.005517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:07.005593 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:07.005588    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:07.005699 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:07.005649    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:09.005149 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:09.005118    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:09.005633 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:09.005127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:09.005633 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:09.005236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:09.005633 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:09.005226    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:09.005633 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:09.005346    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:09.005633 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:09.005426    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:10.669348 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:10.669319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:10.669732 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.669424    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:10.669732 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.669477    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.669463365 +0000 UTC m=+34.089950203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:10.769605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:10.769580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:10.769680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:10.769627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:10.769725 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769712    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:10.769725 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769721    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:10.769785 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769735    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:10.769785 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769743    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:10.769785 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769770    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.769757487 +0000 UTC m=+34.190244329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:10.769874 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:10.769785    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.769778471 +0000 UTC m=+34.190265308 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:11.004453 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:11.004412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:11.004606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:11.004413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:11.004606 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:11.004544    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:11.004718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:11.004418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:11.004718 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:11.004645    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:11.004820 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:11.004749    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:13.005342 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:13.005307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:13.005748 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:13.005416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:13.005748 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:13.005423    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:13.005748 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:13.005447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:13.005748 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:13.005520    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:13.005748 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:13.005611    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:15.004787 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.004628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:15.005643 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.004629    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:15.005643 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:15.004881    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:15.005643 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.004684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:15.005643 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:15.004960    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:15.005643 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:15.005020    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:15.207554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.207524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x" event={"ID":"3ab9fdd5-b061-4357-ae1f-add31fb423ff","Type":"ContainerStarted","Data":"3c604c9646e808aea4befbb0c8122af16371628e594bfdcc405f8dacdffc81d3"}
Sep 16 18:28:15.208918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.208895    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5g7ts" event={"ID":"8eed79a5-6b78-4e0d-8516-cdea9b3f726d","Type":"ContainerStarted","Data":"1bf8e7bd7870b409194388aa881901845e2e6df66bcc980add1581c30a81880a"}
Sep 16 18:28:15.210388 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.210357    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="bd4e51c44522d9d5c4af13c7d4877e2e3cf6830223020d73bacb7046e50d3be6" exitCode=0
Sep 16 18:28:15.210464 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.210437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"bd4e51c44522d9d5c4af13c7d4877e2e3cf6830223020d73bacb7046e50d3be6"}
Sep 16 18:28:15.213010 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.212970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"365505ee31160c58abc2c63ce01680b266d978acbde2a58a1c14003a70851cce"}
Sep 16 18:28:15.213080 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.213037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"7878afa20be6f98708be0a57cbd55aa4746b21419066b98ef2f2656fd1ecd4da"}
Sep 16 18:28:15.213080 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.213074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"3e7259396c44f1d156a237121692fafa0c9917ac04e4390c12746961b7e56157"}
Sep 16 18:28:15.213154 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.213104    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"3331a63cd5ce835b18c6a5f372df1000ad3815d37081899d3dad17f993e25eb1"}
Sep 16 18:28:15.214600 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.214575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x8zxq" event={"ID":"182dea7c-2698-468e-88bc-428ad9033d36","Type":"ContainerStarted","Data":"a3c34db60638dcdc0d1c2cd83d015db9dd0b6755d7c526a6169bc59121846125"}
Sep 16 18:28:15.216096 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.216078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xxksn" event={"ID":"0b06ca00-79eb-4e58-90bb-af7533456332","Type":"ContainerStarted","Data":"41dc9ec4363594fe8b43eb73d04dbef489cb0e70b3156866d2369722ce349fbb"}
Sep 16 18:28:15.218135 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.218091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lg8bf" event={"ID":"c71ae9c8-62cc-4ea2-8dec-b5b5ad214f19","Type":"ContainerStarted","Data":"8fe48d7a615292c0627535de54d5e6e117a3df172b4f1cd120a0ec7928a93c20"}
Sep 16 18:28:15.219515 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.219494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q5qr4" event={"ID":"03db53a4-ede7-4baf-825b-8757e23d1a01","Type":"ContainerStarted","Data":"5b32aafbe2a7439ca54fb4667c7f04f426e8b94f375869428daa33526c8a4ec4"}
Sep 16 18:28:15.228623 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.228585    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" podStartSLOduration=21.2285738 podStartE2EDuration="21.2285738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:58.189391516 +0000 UTC m=+5.609878381" watchObservedRunningTime="2026-09-16 18:28:15.2285738 +0000 UTC m=+22.649060659"
Sep 16 18:28:15.255210 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.255173    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5g7ts" podStartSLOduration=3.848616583 podStartE2EDuration="22.255162687s" totalImagesPullingTime="18.406546104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:15.228686751 +0000 UTC m=+22.649173610" watchObservedRunningTime="2026-09-16 18:28:15.255162687 +0000 UTC m=+22.675649590"
Sep 16 18:28:15.273978 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.273925    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lg8bf" podStartSLOduration=3.799916751 podStartE2EDuration="22.27391382s" totalImagesPullingTime="18.473997069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:15.273859452 +0000 UTC m=+22.694346313" watchObservedRunningTime="2026-09-16 18:28:15.27391382 +0000 UTC m=+22.694400680"
Sep 16 18:28:15.292901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.292871    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q5qr4" podStartSLOduration=3.889759616 podStartE2EDuration="22.292859445s" totalImagesPullingTime="18.403099829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:15.292597831 +0000 UTC m=+22.713084694" watchObservedRunningTime="2026-09-16 18:28:15.292859445 +0000 UTC m=+22.713346306"
Sep 16 18:28:15.307907 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.307870    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xxksn" podStartSLOduration=3.911152638 podStartE2EDuration="22.30785856s" totalImagesPullingTime="18.396705922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:15.307571549 +0000 UTC m=+22.728058410" watchObservedRunningTime="2026-09-16 18:28:15.30785856 +0000 UTC m=+22.728345421"
Sep 16 18:28:15.323829 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:15.323796    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x8zxq" podStartSLOduration=3.911771549 podStartE2EDuration="22.323783732s" totalImagesPullingTime="18.412012183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:15.323696198 +0000 UTC m=+22.744183058" watchObservedRunningTime="2026-09-16 18:28:15.323783732 +0000 UTC m=+22.744270591"
Sep 16 18:28:16.223868 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.223590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjz6z" event={"ID":"f937f5ba-2468-4b6e-8037-e2a4107f754c","Type":"ContainerStarted","Data":"28f51d54686e3fbb9f2846f6c70ace12813f6fa22d47a61d96baa0acecc249a3"}
Sep 16 18:28:16.227318 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.227285    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"59b9519c63a846b79bbf7cb34ed1ec9bd97706517f5018e78a0c6daa54768b50"}
Sep 16 18:28:16.227423 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.227366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"38137886b22c79a501054e41cf48208e44ec71060f32f90b18fdbed0d68ee249"}
Sep 16 18:28:16.805365 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.805339    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:28:16.993792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.993675    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:28:16.805361485Z","UUID":"05df298d-ba4a-47b1-9c39-fe7eff6d11a0","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:28:16.995313 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.995294    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:28:16.995452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:16.995327    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:28:17.005008 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:17.004987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:17.005117 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:17.004994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:17.005182 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:17.005122    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:17.005247 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:17.005230    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:17.005298 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:17.004995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:17.005333 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:17.005313    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:17.231228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:17.231197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x" event={"ID":"3ab9fdd5-b061-4357-ae1f-add31fb423ff","Type":"ContainerStarted","Data":"800ff13057bd6f1888ff32f88945f15cfae5c07fad962cddd25ca24216a57b4f"}
Sep 16 18:28:18.240038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.239997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"5833ed774d22b64bde5488f18a71bdddfffa0d9cf2e71ecb920c8ec6540f1599"}
Sep 16 18:28:18.242306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.242281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x" event={"ID":"3ab9fdd5-b061-4357-ae1f-add31fb423ff","Type":"ContainerStarted","Data":"5304fdd293e94bdfd1a4834d59486c19ca79c68c11797c153ee0b596d55c01e1"}
Sep 16 18:28:18.261498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.261453    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gt8x" podStartSLOduration=3.770847141 podStartE2EDuration="25.261434488s" totalImagesPullingTime="21.490587347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:18.261400325 +0000 UTC m=+25.681887184" watchObservedRunningTime="2026-09-16 18:28:18.261434488 +0000 UTC m=+25.681921350"
Sep 16 18:28:18.261960 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.261909    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bjz6z" podStartSLOduration=6.834552887 podStartE2EDuration="25.261896911s" totalImagesPullingTime="18.427344024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.245689932 +0000 UTC m=+23.666176804" watchObservedRunningTime="2026-09-16 18:28:18.261896911 +0000 UTC m=+25.682383774"
Sep 16 18:28:18.743988 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.743962    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:28:18.744465 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:18.744447    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:28:19.004876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:19.004800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:19.004876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:19.004800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:19.005101 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:19.004924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:19.005101 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:19.004921    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:19.005101 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:19.005017    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:19.005262 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:19.005111    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:21.004335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.004306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:21.004335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.004326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:21.004335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.004334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:21.004767 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:21.004410    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:21.004767 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:21.004457    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:21.004767 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:21.004548    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:21.255633 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.255390    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" event={"ID":"2fb4ca0a-da96-4546-95d9-59121e571d61","Type":"ContainerStarted","Data":"e05e5ea5dcbd9b79b7bec0964a5b0ce82ac184c76d517c971aa2931c4381adec"}
Sep 16 18:28:21.256693 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.256670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:21.256780 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.256761    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:21.256861 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.256850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:21.276009 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.275988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:21.276105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.276095    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:21.291913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:21.291866    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r" podStartSLOduration=9.822368066 podStartE2EDuration="28.29185035s" totalImagesPullingTime="18.469482284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:21.290558234 +0000 UTC m=+28.711045104" watchObservedRunningTime="2026-09-16 18:28:21.29185035 +0000 UTC m=+28.712337210"
Sep 16 18:28:22.259851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.259820    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="ce5132592fb7a443d09b18fef0039737a5bacef917fecff85dfffd6f47fa3ec1" exitCode=0
Sep 16 18:28:22.260559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.259913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"ce5132592fb7a443d09b18fef0039737a5bacef917fecff85dfffd6f47fa3ec1"}
Sep 16 18:28:22.960186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.960004    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5fns9"]
Sep 16 18:28:22.960317 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.960290    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:22.960419 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:22.960399    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:22.962869 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.962840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m49ct"]
Sep 16 18:28:22.963010 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.962978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:22.963111 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:22.963086    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:22.963488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.963466    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fcbbv"]
Sep 16 18:28:22.963564 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:22.963555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:22.963658 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:22.963639    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:23.872739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:23.872672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:28:23.873157 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:23.872831    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 18:28:23.873272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:23.873249    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x8zxq"
Sep 16 18:28:24.270515 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:24.270485    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="caa97c73a7b164ef26faebe460a9dffa978155986f976709d228c38289b72d10" exitCode=0
Sep 16 18:28:24.270676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:24.270566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"caa97c73a7b164ef26faebe460a9dffa978155986f976709d228c38289b72d10"}
Sep 16 18:28:25.004547 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:25.004517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:25.005002 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:25.004624    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:25.005002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:25.004517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:25.005002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:25.004663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:25.005002 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:25.004744    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:25.005002 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:25.004843    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:26.276703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:26.276664    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="6a5a7955b6bbd96a044eae4c28753392d98ba84a79f5b9d4b5c47904c9ba6e0b" exitCode=0
Sep 16 18:28:26.277048 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:26.276704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"6a5a7955b6bbd96a044eae4c28753392d98ba84a79f5b9d4b5c47904c9ba6e0b"}
Sep 16 18:28:26.693675 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:26.693649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:26.693806 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.693755    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:26.693806 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.693804    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs podName:80d732e2-c418-40e7-bd17-6258bf894253 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.693790625 +0000 UTC m=+66.114277462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs") pod "network-metrics-daemon-m49ct" (UID: "80d732e2-c418-40e7-bd17-6258bf894253") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:26.794208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:26.794174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:26.794372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:26.794244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:26.794372 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794322    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:26.794491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794375    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:26.794491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794396    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:26.794491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794409    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4k22z for pod openshift-network-diagnostics/network-check-target-fcbbv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:26.794491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794396    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret podName:7c049995-694f-4ace-b1a7-6ea9f6d047a7 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.794377639 +0000 UTC m=+66.214864494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret") pod "global-pull-secret-syncer-5fns9" (UID: "7c049995-694f-4ace-b1a7-6ea9f6d047a7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:26.794491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:26.794462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z podName:83ca583e-a760-4868-b418-a76b2f4b7a09 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.794448739 +0000 UTC m=+66.214935577 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4k22z" (UniqueName: "kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z") pod "network-check-target-fcbbv" (UID: "83ca583e-a760-4868-b418-a76b2f4b7a09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:27.004548 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.004483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:27.004683 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.004483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:27.004683 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:27.004586    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fcbbv" podUID="83ca583e-a760-4868-b418-a76b2f4b7a09"
Sep 16 18:28:27.004683 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:27.004642    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5fns9" podUID="7c049995-694f-4ace-b1a7-6ea9f6d047a7"
Sep 16 18:28:27.004683 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.004494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:27.004816 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:27.004733    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m49ct" podUID="80d732e2-c418-40e7-bd17-6258bf894253"
Sep 16 18:28:27.356159 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.356093    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeReady"
Sep 16 18:28:27.356546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.356233    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:28:27.406720 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.406686    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lrwf8"]
Sep 16 18:28:27.419590 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.419564    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-864pq"]
Sep 16 18:28:27.419705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.419689    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.422690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.422664    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:28:27.422831 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.422771    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sr6v8\""
Sep 16 18:28:27.422831 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.422806    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:28:27.428573 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.428545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lrwf8"]
Sep 16 18:28:27.428709 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.428590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-864pq"]
Sep 16 18:28:27.428709 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.428679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.432214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.430895    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:28:27.432214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.431130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:28:27.432214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.431173    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:28:27.432214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.431130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g75hp\""
Sep 16 18:28:27.601423 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a8ec31e-9939-4934-87a8-5119122e5001-cert\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.601423 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwwvk\" (UniqueName: \"kubernetes.io/projected/1a8ec31e-9939-4934-87a8-5119122e5001-kube-api-access-nwwvk\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.601681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.601681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601485    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknfs\" (UniqueName: \"kubernetes.io/projected/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-kube-api-access-qknfs\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.601681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601536    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-config-volume\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.601681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.601594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-tmp-dir\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.702661 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.702630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.702802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.702677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qknfs\" (UniqueName: \"kubernetes.io/projected/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-kube-api-access-qknfs\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.702802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.702715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-config-volume\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.702802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.702743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-tmp-dir\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.702802 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:27.702787    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:27.703027 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:27.702854    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:28.202834318 +0000 UTC m=+35.623321175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:27.703090 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.703038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a8ec31e-9939-4934-87a8-5119122e5001-cert\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.703090 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.703071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwwvk\" (UniqueName: \"kubernetes.io/projected/1a8ec31e-9939-4934-87a8-5119122e5001-kube-api-access-nwwvk\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.703608 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.703586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-tmp-dir\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.703817 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.703800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-config-volume\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.709094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.708971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a8ec31e-9939-4934-87a8-5119122e5001-cert\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.715507 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.715483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknfs\" (UniqueName: \"kubernetes.io/projected/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-kube-api-access-qknfs\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:27.717478 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.717459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwwvk\" (UniqueName: \"kubernetes.io/projected/1a8ec31e-9939-4934-87a8-5119122e5001-kube-api-access-nwwvk\") pod \"ingress-canary-864pq\" (UID: \"1a8ec31e-9939-4934-87a8-5119122e5001\") " pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:27.746431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:27.746406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-864pq"
Sep 16 18:28:28.064723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:28.064693    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-864pq"]
Sep 16 18:28:28.068542 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:28.068518    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8ec31e_9939_4934_87a8_5119122e5001.slice/crio-74c4d0cc86802e4190fc0ec8ee2ac848a77eb403683f8a66ac6310b4d0b57fa6 WatchSource:0}: Error finding container 74c4d0cc86802e4190fc0ec8ee2ac848a77eb403683f8a66ac6310b4d0b57fa6: Status 404 returned error can't find the container with id 74c4d0cc86802e4190fc0ec8ee2ac848a77eb403683f8a66ac6310b4d0b57fa6
Sep 16 18:28:28.207399 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:28.207371    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:28.207578 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:28.207491    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:28.207578 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:28.207555    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:29.207534724 +0000 UTC m=+36.628021569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:28.282907 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:28.282876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-864pq" event={"ID":"1a8ec31e-9939-4934-87a8-5119122e5001","Type":"ContainerStarted","Data":"74c4d0cc86802e4190fc0ec8ee2ac848a77eb403683f8a66ac6310b4d0b57fa6"}
Sep 16 18:28:29.005214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.005177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:29.006093 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.005177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:29.006093 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.005176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:29.008852 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.008827    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:28:29.009002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.008977    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:28:29.009276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.009257    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:28:29.009507 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.009492    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:28:29.009833 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.009812    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw4s4\""
Sep 16 18:28:29.009960 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.009817    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9wz2\""
Sep 16 18:28:29.215113 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:29.215070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:29.215268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:29.215174    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:29.215268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:29.215234    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:31.215217673 +0000 UTC m=+38.635704526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:30.939825 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.939796    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"]
Sep 16 18:28:30.952463 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.952440    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"]
Sep 16 18:28:30.952603 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.952536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:30.956680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.956648    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 18:28:30.956805 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.956686    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:28:30.956805 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.956741    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-crvdp\""
Sep 16 18:28:30.957918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.957004    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:28:30.957918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:30.957229    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 18:28:31.029106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.029082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a5c79871-1520-4581-8a74-d4c1e6ca07d3-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.029106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.029109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.029301 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.029145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g2vs\" (UniqueName: \"kubernetes.io/projected/a5c79871-1520-4581-8a74-d4c1e6ca07d3-kube-api-access-8g2vs\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.051506 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.051485    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"]
Sep 16 18:28:31.058307 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.058282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"
Sep 16 18:28:31.063600 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.063580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ptf84\""
Sep 16 18:28:31.063881 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.063859    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-czsgb"]
Sep 16 18:28:31.070643 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.070625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.073552 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.073531    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 18:28:31.073912 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.073893    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 18:28:31.074150 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.074131    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:28:31.074244 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.074202    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nnm98\""
Sep 16 18:28:31.074466 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.074450    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:28:31.078365 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.078336    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"]
Sep 16 18:28:31.083211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.083190    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 18:28:31.083299 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.083244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-czsgb"]
Sep 16 18:28:31.129855 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.129825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a5c79871-1520-4581-8a74-d4c1e6ca07d3-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.130015 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.129873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.130015 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.129903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef541df6-cc2e-4d36-b932-392a6411733b-serving-cert\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130015 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.130006    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:31.130178 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-service-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130178 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.130064    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:31.63004953 +0000 UTC m=+39.050536368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:31.130178 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-tmp\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130178 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g2vs\" (UniqueName: \"kubernetes.io/projected/a5c79871-1520-4581-8a74-d4c1e6ca07d3-kube-api-access-8g2vs\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.130404 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130404 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-snapshots\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130404 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjhb\" (UniqueName: \"kubernetes.io/projected/8801d24a-3a55-4038-a098-922085e8ac77-kube-api-access-npjhb\") pod \"network-check-source-6f8b768455-zhk4q\" (UID: \"8801d24a-3a55-4038-a098-922085e8ac77\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"
Sep 16 18:28:31.130404 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wbw\" (UniqueName: \"kubernetes.io/projected/ef541df6-cc2e-4d36-b932-392a6411733b-kube-api-access-x5wbw\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.130716 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.130697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a5c79871-1520-4581-8a74-d4c1e6ca07d3-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.152854 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.152824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g2vs\" (UniqueName: \"kubernetes.io/projected/a5c79871-1520-4581-8a74-d4c1e6ca07d3-kube-api-access-8g2vs\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.153569 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.153545    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bcfdf658d-mlcc6"]
Sep 16 18:28:31.162641 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.162623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.166834 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.166816    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 18:28:31.171543 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.171524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 18:28:31.172680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.172652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jnwzm\""
Sep 16 18:28:31.174534 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.174515    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 18:28:31.183817 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.183796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bcfdf658d-mlcc6"]
Sep 16 18:28:31.186220 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.186202    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 18:28:31.231346 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjhb\" (UniqueName: \"kubernetes.io/projected/8801d24a-3a55-4038-a098-922085e8ac77-kube-api-access-npjhb\") pod \"network-check-source-6f8b768455-zhk4q\" (UID: \"8801d24a-3a55-4038-a098-922085e8ac77\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"
Sep 16 18:28:31.231346 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wbw\" (UniqueName: \"kubernetes.io/projected/ef541df6-cc2e-4d36-b932-392a6411733b-kube-api-access-x5wbw\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.231523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.231523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:31.231523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef541df6-cc2e-4d36-b932-392a6411733b-serving-cert\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.231523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-service-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.231523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-tmp\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.231568    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.231633    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:35.231614485 +0000 UTC m=+42.652101342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.231757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9lrq\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.232094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-snapshots\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.232094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.232094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.232094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-tmp\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.232094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.231887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.232256 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.232155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-service-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.232505 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.232483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef541df6-cc2e-4d36-b932-392a6411733b-snapshots\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.232560 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.232512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef541df6-cc2e-4d36-b932-392a6411733b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.234125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.234101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef541df6-cc2e-4d36-b932-392a6411733b-serving-cert\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.243905 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.243882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wbw\" (UniqueName: \"kubernetes.io/projected/ef541df6-cc2e-4d36-b932-392a6411733b-kube-api-access-x5wbw\") pod \"insights-operator-5844b856db-czsgb\" (UID: \"ef541df6-cc2e-4d36-b932-392a6411733b\") " pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.245331 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.245311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjhb\" (UniqueName: \"kubernetes.io/projected/8801d24a-3a55-4038-a098-922085e8ac77-kube-api-access-npjhb\") pod \"network-check-source-6f8b768455-zhk4q\" (UID: \"8801d24a-3a55-4038-a098-922085e8ac77\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"
Sep 16 18:28:31.332827 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.332827 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.332993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.333016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.333038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9lrq\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333365 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.333224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.333421 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.333365    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:31.333421 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.333384    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:31.333534 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.333456    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:31.833435287 +0000 UTC m=+39.253922135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:31.333770 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.333741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.334002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.333980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.335608 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.335581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.335992 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.335961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.342418 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.342397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.343513 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.343492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9lrq\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.370331 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.370308    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"
Sep 16 18:28:31.381473 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.381455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-czsgb"
Sep 16 18:28:31.634324 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.634291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:31.634459 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.634441    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:31.634515 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.634506    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:32.634491432 +0000 UTC m=+40.054978269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:31.836589 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:31.836559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:31.836718 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.836702    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:31.836754 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.836720    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:31.836788 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:31.836769    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:32.836755513 +0000 UTC m=+40.257242351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:32.297485 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.297162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerStarted","Data":"1d0c1d8bb4114b928063e718538b0fc3d6b2e6a91188cbb50730b168c9637120"}
Sep 16 18:28:32.298959 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.298902    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-864pq" event={"ID":"1a8ec31e-9939-4934-87a8-5119122e5001","Type":"ContainerStarted","Data":"91eba4d6e57037798a8bee4707063aab8399e970bf666638d1272f38363be5d9"}
Sep 16 18:28:32.340428 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.340405    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-czsgb"]
Sep 16 18:28:32.341617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.341372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q"]
Sep 16 18:28:32.394029 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:32.393996    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef541df6_cc2e_4d36_b932_392a6411733b.slice/crio-6d8162441edda09e74afadd32d7ea44a332398ab0e45cdff454994de4dbcda60 WatchSource:0}: Error finding container 6d8162441edda09e74afadd32d7ea44a332398ab0e45cdff454994de4dbcda60: Status 404 returned error can't find the container with id 6d8162441edda09e74afadd32d7ea44a332398ab0e45cdff454994de4dbcda60
Sep 16 18:28:32.394278 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:32.394252    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8801d24a_3a55_4038_a098_922085e8ac77.slice/crio-cca4a44202c067dd3196e3494b87b7c9f7f4760a9ef8b5f8c09e20e38698c20e WatchSource:0}: Error finding container cca4a44202c067dd3196e3494b87b7c9f7f4760a9ef8b5f8c09e20e38698c20e: Status 404 returned error can't find the container with id cca4a44202c067dd3196e3494b87b7c9f7f4760a9ef8b5f8c09e20e38698c20e
Sep 16 18:28:32.643320 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.643252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:32.643424 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:32.643368    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:32.643463 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:32.643427    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:34.643413922 +0000 UTC m=+42.063900760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:32.845107 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:32.845073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:32.845239 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:32.845224    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:32.845285 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:32.845242    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:32.845319 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:32.845288    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:34.845273676 +0000 UTC m=+42.265760514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:33.308646 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:33.308555    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="1d0c1d8bb4114b928063e718538b0fc3d6b2e6a91188cbb50730b168c9637120" exitCode=0
Sep 16 18:28:33.309048 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:33.308955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"1d0c1d8bb4114b928063e718538b0fc3d6b2e6a91188cbb50730b168c9637120"}
Sep 16 18:28:33.311742 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:33.311684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-czsgb" event={"ID":"ef541df6-cc2e-4d36-b932-392a6411733b","Type":"ContainerStarted","Data":"6d8162441edda09e74afadd32d7ea44a332398ab0e45cdff454994de4dbcda60"}
Sep 16 18:28:33.313193 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:33.313167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q" event={"ID":"8801d24a-3a55-4038-a098-922085e8ac77","Type":"ContainerStarted","Data":"cca4a44202c067dd3196e3494b87b7c9f7f4760a9ef8b5f8c09e20e38698c20e"}
Sep 16 18:28:33.334647 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:33.334601    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-864pq" podStartSLOduration=2.327796333 podStartE2EDuration="6.334585232s" totalImagesPullingTime="4.006788899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:32.351350476 +0000 UTC m=+39.771837337" watchObservedRunningTime="2026-09-16 18:28:33.334585232 +0000 UTC m=+40.755072092"
Sep 16 18:28:34.321110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:34.320917    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c5b3366-d2a1-4cd2-90d1-c207b8412337" containerID="4b6c86c1cb2d35822cb6a0b69e78e8fe409f002d5b2200d8be337624874e1b82" exitCode=0
Sep 16 18:28:34.321110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:34.321001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerDied","Data":"4b6c86c1cb2d35822cb6a0b69e78e8fe409f002d5b2200d8be337624874e1b82"}
Sep 16 18:28:34.661856 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:34.661824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:34.661991 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:34.661976    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:34.662048 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:34.662033    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.662014793 +0000 UTC m=+46.082501646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:34.863159 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:34.863098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:34.863268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:34.863234    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:34.863268 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:34.863252    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:34.863333 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:34.863307    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.86329277 +0000 UTC m=+46.283779612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:35.266322 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:35.266278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:35.266516 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:35.266452    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:35.266587 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:35.266527    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:43.266507286 +0000 UTC m=+50.686994126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:35.327412 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:35.327376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" event={"ID":"4c5b3366-d2a1-4cd2-90d1-c207b8412337","Type":"ContainerStarted","Data":"1a792d77cc30e7c59f0a050c2c0f68a9e3c377421b800ca21cfd818a9a9b0a24"}
Sep 16 18:28:35.350925 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:35.350861    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m4bmk" podStartSLOduration=9.806603452000001 podStartE2EDuration="42.350841609s" totalImagesPullingTime="32.544238157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:35.349442773 +0000 UTC m=+42.769929633" watchObservedRunningTime="2026-09-16 18:28:35.350841609 +0000 UTC m=+42.771328470"
Sep 16 18:28:36.331874 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:36.331850    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-czsgb" event={"ID":"ef541df6-cc2e-4d36-b932-392a6411733b","Type":"ContainerStarted","Data":"43061decc38190f39188cb60cae55369e76b100d999e7a5698dbb48cdd145d5c"}
Sep 16 18:28:36.357480 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:36.357437    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-czsgb" podStartSLOduration=1.800248211 podStartE2EDuration="5.357421528s" totalImagesPullingTime="3.557173317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:36.356252663 +0000 UTC m=+43.776739523" watchObservedRunningTime="2026-09-16 18:28:36.357421528 +0000 UTC m=+43.777908378"
Sep 16 18:28:37.335715 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.335680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q" event={"ID":"8801d24a-3a55-4038-a098-922085e8ac77","Type":"ContainerStarted","Data":"75c22f8586982e235c90e8c399798af3bd9cdb7615a782e136bc088d006a287f"}
Sep 16 18:28:37.354282 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.354236    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-zhk4q" podStartSLOduration=2.457924529 podStartE2EDuration="6.354220269s" totalImagesPullingTime="3.89629574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:37.353464294 +0000 UTC m=+44.773951156" watchObservedRunningTime="2026-09-16 18:28:37.354220269 +0000 UTC m=+44.774707129"
Sep 16 18:28:37.391222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.388492    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"]
Sep 16 18:28:37.416656 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.416633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"]
Sep 16 18:28:37.416761 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.416712    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"
Sep 16 18:28:37.419269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.419250    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hffr8\""
Sep 16 18:28:37.419269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.419264    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 16 18:28:37.419430 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.419301    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 16 18:28:37.483240 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.483216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnxsw\" (UniqueName: \"kubernetes.io/projected/6a52dbd8-8c64-497c-8ed6-38a536080c20-kube-api-access-wnxsw\") pod \"migrator-8b4d697c5-wmbj5\" (UID: \"6a52dbd8-8c64-497c-8ed6-38a536080c20\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"
Sep 16 18:28:37.583695 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.583661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnxsw\" (UniqueName: \"kubernetes.io/projected/6a52dbd8-8c64-497c-8ed6-38a536080c20-kube-api-access-wnxsw\") pod \"migrator-8b4d697c5-wmbj5\" (UID: \"6a52dbd8-8c64-497c-8ed6-38a536080c20\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"
Sep 16 18:28:37.598338 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.598282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnxsw\" (UniqueName: \"kubernetes.io/projected/6a52dbd8-8c64-497c-8ed6-38a536080c20-kube-api-access-wnxsw\") pod \"migrator-8b4d697c5-wmbj5\" (UID: \"6a52dbd8-8c64-497c-8ed6-38a536080c20\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"
Sep 16 18:28:37.600564 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.600535    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vqhhh"]
Sep 16 18:28:37.615280 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.615261    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqhhh"]
Sep 16 18:28:37.615383 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.615352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.617959 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.617919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:28:37.618078 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.618011    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5rq9\""
Sep 16 18:28:37.618140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.617943    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:28:37.684458 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.684437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.684594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.684481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd37ec71-986a-45dc-ae05-4bdd19df7929-crio-socket\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.684594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.684506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.684708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.684605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd37ec71-986a-45dc-ae05-4bdd19df7929-data-volume\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.684708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.684650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bh98\" (UniqueName: \"kubernetes.io/projected/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-api-access-9bh98\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.724645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.724625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"
Sep 16 18:28:37.786254 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.786012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd37ec71-986a-45dc-ae05-4bdd19df7929-data-volume\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.786254 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.786085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bh98\" (UniqueName: \"kubernetes.io/projected/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-api-access-9bh98\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.786254 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.786122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.786254 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.786173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd37ec71-986a-45dc-ae05-4bdd19df7929-crio-socket\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.786254 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.786213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.786516 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:37.786338    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:37.786516 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:37.786419    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.286399419 +0000 UTC m=+45.706886260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:37.787339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.787019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd37ec71-986a-45dc-ae05-4bdd19df7929-data-volume\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.787339 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:37.787039    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:37.787339 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:37.787098    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.287082132 +0000 UTC m=+45.707568971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:37.787339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.787172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd37ec71-986a-45dc-ae05-4bdd19df7929-crio-socket\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:37.803766 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:37.803520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bh98\" (UniqueName: \"kubernetes.io/projected/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-api-access-9bh98\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:38.005821 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.005749    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5"]
Sep 16 18:28:38.008315 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:38.008287    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a52dbd8_8c64_497c_8ed6_38a536080c20.slice/crio-408498057f0502b8513a5f4cf72040bd9fa1ce3847f99c44cd1832a827a99179 WatchSource:0}: Error finding container 408498057f0502b8513a5f4cf72040bd9fa1ce3847f99c44cd1832a827a99179: Status 404 returned error can't find the container with id 408498057f0502b8513a5f4cf72040bd9fa1ce3847f99c44cd1832a827a99179
Sep 16 18:28:38.289797 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.289732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:38.289797 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.289786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:38.289996 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.289885    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:38.289996 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.289970    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:39.289953498 +0000 UTC m=+46.710440349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:38.289996 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.289886    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:38.290094 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.290028    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:39.290014776 +0000 UTC m=+46.710501617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:38.339702 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.339680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5" event={"ID":"6a52dbd8-8c64-497c-8ed6-38a536080c20","Type":"ContainerStarted","Data":"408498057f0502b8513a5f4cf72040bd9fa1ce3847f99c44cd1832a827a99179"}
Sep 16 18:28:38.692035 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.692007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:38.692187 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.692166    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:38.692250 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.692243    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:46.692224983 +0000 UTC m=+54.112711823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:38.894127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:38.894088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:38.894288 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.894237    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:38.894288 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.894257    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:38.894413 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:38.894320    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:46.894300715 +0000 UTC m=+54.314787558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:39.297092 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:39.297055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:39.297281 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:39.297137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:39.297281 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:39.297221    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:39.297281 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:39.297233    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:39.297419 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:39.297289    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:41.297271353 +0000 UTC m=+48.717758190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:39.297419 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:39.297303    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:41.297296205 +0000 UTC m=+48.717783043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:40.345360 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:40.345327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5" event={"ID":"6a52dbd8-8c64-497c-8ed6-38a536080c20","Type":"ContainerStarted","Data":"ea53e7e91da6456388f5e6755b7b112eda68191ee8702a408b4905ef8525c6ff"}
Sep 16 18:28:40.345722 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:40.345377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5" event={"ID":"6a52dbd8-8c64-497c-8ed6-38a536080c20","Type":"ContainerStarted","Data":"ed8919273357b12d114c37fd000edb004a2b1f91d035fac80bbca7d29674ba1d"}
Sep 16 18:28:40.363042 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:40.362996    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wmbj5" podStartSLOduration=1.6333016740000001 podStartE2EDuration="3.362981132s" totalImagesPullingTime="1.729679458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:40.361474756 +0000 UTC m=+47.781961617" watchObservedRunningTime="2026-09-16 18:28:40.362981132 +0000 UTC m=+47.783467992"
Sep 16 18:28:40.502236 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:40.502211    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xxksn_0b06ca00-79eb-4e58-90bb-af7533456332/dns-node-resolver/0.log"
Sep 16 18:28:41.302000 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:41.301974    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5g7ts_8eed79a5-6b78-4e0d-8516-cdea9b3f726d/node-ca/0.log"
Sep 16 18:28:41.314295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:41.314266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:41.314401 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:41.314318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:41.314401 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:41.314394    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:41.314491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:41.314418    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:41.314491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:41.314446    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:45.314432607 +0000 UTC m=+52.734919445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:41.314491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:41.314464    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:45.314451172 +0000 UTC m=+52.734938009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:42.105776 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:42.105747    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-864pq_1a8ec31e-9939-4934-87a8-5119122e5001/serve-healthcheck-canary/0.log"
Sep 16 18:28:42.502709 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:42.502681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wmbj5_6a52dbd8-8c64-497c-8ed6-38a536080c20/migrator/0.log"
Sep 16 18:28:42.702691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:42.702664    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wmbj5_6a52dbd8-8c64-497c-8ed6-38a536080c20/graceful-termination/0.log"
Sep 16 18:28:43.329056 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:43.329013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:43.329426 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:43.329164    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:43.329426 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:43.329228    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls podName:b69b3521-a79f-44b0-99d8-9dc6d045f3c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:59.32920885 +0000 UTC m=+66.749695688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls") pod "dns-default-lrwf8" (UID: "b69b3521-a79f-44b0-99d8-9dc6d045f3c4") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:45.344917 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:45.344883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:45.345425 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:45.345001    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:45.345425 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:45.345065    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:53.34504826 +0000 UTC m=+60.765535146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:45.345425 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:45.345009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:45.345425 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:45.345085    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:45.345425 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:45.345141    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls podName:dd37ec71-986a-45dc-ae05-4bdd19df7929 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:53.345125934 +0000 UTC m=+60.765612777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqhhh" (UID: "dd37ec71-986a-45dc-ae05-4bdd19df7929") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:46.755983 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:46.755944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:28:46.756424 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:46.756068    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:46.756424 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:46.756122    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls podName:a5c79871-1520-4581-8a74-d4c1e6ca07d3 nodeName:}" failed. No retries permitted until 2026-09-16 18:29:02.756108137 +0000 UTC m=+70.176594995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mlb96" (UID: "a5c79871-1520-4581-8a74-d4c1e6ca07d3") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:28:46.957202 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:46.957167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") pod \"image-registry-6bcfdf658d-mlcc6\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") " pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:28:46.957336 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:46.957310    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:28:46.957336 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:46.957326    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bcfdf658d-mlcc6: secret "image-registry-tls" not found
Sep 16 18:28:46.957418 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:46.957379    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls podName:cca8f41f-849f-4c4d-aafb-1b097deabf52 nodeName:}" failed. No retries permitted until 2026-09-16 18:29:02.957364647 +0000 UTC m=+70.377851489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls") pod "image-registry-6bcfdf658d-mlcc6" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52") : secret "image-registry-tls" not found
Sep 16 18:28:53.279756 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.279726    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gm67r"
Sep 16 18:28:53.402272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.402248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:53.402538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.402491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:53.402793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.402774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd37ec71-986a-45dc-ae05-4bdd19df7929-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:53.404657 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.404638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd37ec71-986a-45dc-ae05-4bdd19df7929-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqhhh\" (UID: \"dd37ec71-986a-45dc-ae05-4bdd19df7929\") " pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:53.526206 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.526184    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5rq9\""
Sep 16 18:28:53.534245 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.534209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqhhh"
Sep 16 18:28:53.834457 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:53.834374    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqhhh"]
Sep 16 18:28:53.837402 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:53.837370    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd37ec71_986a_45dc_ae05_4bdd19df7929.slice/crio-752a48be83d83d0d11f3d5441df4cdc52ed840682a0c4a4f607014f14f49db85 WatchSource:0}: Error finding container 752a48be83d83d0d11f3d5441df4cdc52ed840682a0c4a4f607014f14f49db85: Status 404 returned error can't find the container with id 752a48be83d83d0d11f3d5441df4cdc52ed840682a0c4a4f607014f14f49db85
Sep 16 18:28:54.390176 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:54.390142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqhhh" event={"ID":"dd37ec71-986a-45dc-ae05-4bdd19df7929","Type":"ContainerStarted","Data":"0cbef8e86c94a6139f0bb386c606da4472eb0e8447c02306f733967023b24983"}
Sep 16 18:28:54.390538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:54.390219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqhhh" event={"ID":"dd37ec71-986a-45dc-ae05-4bdd19df7929","Type":"ContainerStarted","Data":"752a48be83d83d0d11f3d5441df4cdc52ed840682a0c4a4f607014f14f49db85"}
Sep 16 18:28:55.395578 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:55.395541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqhhh" event={"ID":"dd37ec71-986a-45dc-ae05-4bdd19df7929","Type":"ContainerStarted","Data":"22da1127d1bd33dda1ba985b41bb5fdcfbe85c454e3c420d5a65a264d0a69937"}
Sep 16 18:28:56.400578 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:56.400550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqhhh" event={"ID":"dd37ec71-986a-45dc-ae05-4bdd19df7929","Type":"ContainerStarted","Data":"ff77c6a6677887de4969d032eebafb485aa5190fbcba962cf8ac71ccf850369d"}
Sep 16 18:28:56.426777 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:56.426711    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vqhhh" podStartSLOduration=17.24601347 podStartE2EDuration="19.426695076s" totalImagesPullingTime="2.180681606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:28:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:56.426012558 +0000 UTC m=+63.846499420" watchObservedRunningTime="2026-09-16 18:28:56.426695076 +0000 UTC m=+63.847181935"
Sep 16 18:28:58.745242 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.745195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:58.747647 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.747624    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:28:58.758031 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.757996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d732e2-c418-40e7-bd17-6258bf894253-metrics-certs\") pod \"network-metrics-daemon-m49ct\" (UID: \"80d732e2-c418-40e7-bd17-6258bf894253\") " pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:58.845508 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.845474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:58.845508 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.845516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:58.847808 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.847787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:28:58.848113 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.848094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k22z\" (UniqueName: \"kubernetes.io/projected/83ca583e-a760-4868-b418-a76b2f4b7a09-kube-api-access-4k22z\") pod \"network-check-target-fcbbv\" (UID: \"83ca583e-a760-4868-b418-a76b2f4b7a09\") " pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:58.857949 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:58.857905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c049995-694f-4ace-b1a7-6ea9f6d047a7-original-pull-secret\") pod \"global-pull-secret-syncer-5fns9\" (UID: \"7c049995-694f-4ace-b1a7-6ea9f6d047a7\") " pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:59.019262 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.019197    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw4s4\""
Sep 16 18:28:59.025715 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.025692    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9wz2\""
Sep 16 18:28:59.027483 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.027469    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:59.028599 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.028582    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5fns9"
Sep 16 18:28:59.034014 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.033994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m49ct"
Sep 16 18:28:59.300287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.300267    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fcbbv"]
Sep 16 18:28:59.302538 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:59.302512    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ca583e_a760_4868_b418_a76b2f4b7a09.slice/crio-6f0bece381945cb78a989343ff726c64671dddaa6a4f56f1eec50026f46c3d6f WatchSource:0}: Error finding container 6f0bece381945cb78a989343ff726c64671dddaa6a4f56f1eec50026f46c3d6f: Status 404 returned error can't find the container with id 6f0bece381945cb78a989343ff726c64671dddaa6a4f56f1eec50026f46c3d6f
Sep 16 18:28:59.324645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.324622    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m49ct"]
Sep 16 18:28:59.327101 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.327079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5fns9"]
Sep 16 18:28:59.328953 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:59.328915    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d732e2_c418_40e7_bd17_6258bf894253.slice/crio-92dfd8c2df375068f51a5e5087c1638229467f8d57b480525f0303392e53c659 WatchSource:0}: Error finding container 92dfd8c2df375068f51a5e5087c1638229467f8d57b480525f0303392e53c659: Status 404 returned error can't find the container with id 92dfd8c2df375068f51a5e5087c1638229467f8d57b480525f0303392e53c659
Sep 16 18:28:59.329905 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:59.329867    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c049995_694f_4ace_b1a7_6ea9f6d047a7.slice/crio-892a89313cf042024792771c38105296e13732b7b0529e78c8cba2486e9023b3 WatchSource:0}: Error finding container 892a89313cf042024792771c38105296e13732b7b0529e78c8cba2486e9023b3: Status 404 returned error can't find the container with id 892a89313cf042024792771c38105296e13732b7b0529e78c8cba2486e9023b3
Sep 16 18:28:59.349762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.349739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:59.352083 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.352064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b69b3521-a79f-44b0-99d8-9dc6d045f3c4-metrics-tls\") pod \"dns-default-lrwf8\" (UID: \"b69b3521-a79f-44b0-99d8-9dc6d045f3c4\") " pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:59.410551 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.410522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m49ct" event={"ID":"80d732e2-c418-40e7-bd17-6258bf894253","Type":"ContainerStarted","Data":"92dfd8c2df375068f51a5e5087c1638229467f8d57b480525f0303392e53c659"}
Sep 16 18:28:59.412033 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.412012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fcbbv" event={"ID":"83ca583e-a760-4868-b418-a76b2f4b7a09","Type":"ContainerStarted","Data":"d617956cfc59aa515b9c90535e796e107b8e7387217bb3016947ef3f8edf1c81"}
Sep 16 18:28:59.412104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.412050    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fcbbv" event={"ID":"83ca583e-a760-4868-b418-a76b2f4b7a09","Type":"ContainerStarted","Data":"6f0bece381945cb78a989343ff726c64671dddaa6a4f56f1eec50026f46c3d6f"}
Sep 16 18:28:59.412104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.412096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:28:59.413106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.413087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5fns9" event={"ID":"7c049995-694f-4ace-b1a7-6ea9f6d047a7","Type":"ContainerStarted","Data":"892a89313cf042024792771c38105296e13732b7b0529e78c8cba2486e9023b3"}
Sep 16 18:28:59.429918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.429877    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fcbbv" podStartSLOduration=66.429860605 podStartE2EDuration="1m6.429860605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:59.428872515 +0000 UTC m=+66.849359375" watchObservedRunningTime="2026-09-16 18:28:59.429860605 +0000 UTC m=+66.850347461"
Sep 16 18:28:59.485243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.485217    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bcfdf658d-mlcc6"]
Sep 16 18:28:59.485439 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:28:59.485416    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6" podUID="cca8f41f-849f-4c4d-aafb-1b097deabf52"
Sep 16 18:28:59.534694 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.534673    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sr6v8\""
Sep 16 18:28:59.543468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.543447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:28:59.550297 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.550254    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d89597d46-nnpch"]
Sep 16 18:28:59.555079 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.555061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.572674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.572651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d89597d46-nnpch"]
Sep 16 18:28:59.651579 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-registry-tls\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651702 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7rn\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-kube-api-access-jx7rn\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651702 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-installation-pull-secrets\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651702 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-bound-sa-token\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-trusted-ca\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6279a734-dcbb-4640-9912-60eeddc3144f-ca-trust-extracted\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-registry-certificates\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.651960 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.651910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-image-registry-private-configuration\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753025 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-registry-tls\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7rn\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-kube-api-access-jx7rn\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-installation-pull-secrets\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-bound-sa-token\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-trusted-ca\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6279a734-dcbb-4640-9912-60eeddc3144f-ca-trust-extracted\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-registry-certificates\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753341 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-image-registry-private-configuration\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.753737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.753712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6279a734-dcbb-4640-9912-60eeddc3144f-ca-trust-extracted\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.754072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.754050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-registry-certificates\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.754437 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.754413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6279a734-dcbb-4640-9912-60eeddc3144f-trusted-ca\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.755597 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.755579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-installation-pull-secrets\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.755868 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.755851    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6279a734-dcbb-4640-9912-60eeddc3144f-image-registry-private-configuration\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.756003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.755987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-registry-tls\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.765160 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.765136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-bound-sa-token\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.765576 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.765554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7rn\" (UniqueName: \"kubernetes.io/projected/6279a734-dcbb-4640-9912-60eeddc3144f-kube-api-access-jx7rn\") pod \"image-registry-6d89597d46-nnpch\" (UID: \"6279a734-dcbb-4640-9912-60eeddc3144f\") " pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:28:59.818783 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.818664    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lrwf8"]
Sep 16 18:28:59.821486 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:28:59.821462    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69b3521_a79f_44b0_99d8_9dc6d045f3c4.slice/crio-e5e1ddf28eacc5d338286c2b47c96f1b88b64157e6c4f3cfddc55e51a8724ffb WatchSource:0}: Error finding container e5e1ddf28eacc5d338286c2b47c96f1b88b64157e6c4f3cfddc55e51a8724ffb: Status 404 returned error can't find the container with id e5e1ddf28eacc5d338286c2b47c96f1b88b64157e6c4f3cfddc55e51a8724ffb
Sep 16 18:28:59.866171 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.866151    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jnwzm\""
Sep 16 18:28:59.874166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:28:59.874134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:29:00.171726 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.171695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d89597d46-nnpch"]
Sep 16 18:29:00.279293 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:00.279261    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6279a734_dcbb_4640_9912_60eeddc3144f.slice/crio-2e94b1ad297e6611e19f14d5e0541389310ca7ecd33406d98ff2dc4896b629ec WatchSource:0}: Error finding container 2e94b1ad297e6611e19f14d5e0541389310ca7ecd33406d98ff2dc4896b629ec: Status 404 returned error can't find the container with id 2e94b1ad297e6611e19f14d5e0541389310ca7ecd33406d98ff2dc4896b629ec
Sep 16 18:29:00.421382 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.421341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m49ct" event={"ID":"80d732e2-c418-40e7-bd17-6258bf894253","Type":"ContainerStarted","Data":"f741002bbb0def3d81b4e2c2ec892d7e19b48f1f211ed75cab74b936cac0a754"}
Sep 16 18:29:00.424190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.424123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d89597d46-nnpch" event={"ID":"6279a734-dcbb-4640-9912-60eeddc3144f","Type":"ContainerStarted","Data":"ac49c84c3b7f5fc40b41fd34c7ff4d26de595509c279b2329fd81edca3a21da9"}
Sep 16 18:29:00.424190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.424164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d89597d46-nnpch" event={"ID":"6279a734-dcbb-4640-9912-60eeddc3144f","Type":"ContainerStarted","Data":"2e94b1ad297e6611e19f14d5e0541389310ca7ecd33406d98ff2dc4896b629ec"}
Sep 16 18:29:00.424477 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.424457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:29:00.426030 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.425863    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:29:00.426030 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.425998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrwf8" event={"ID":"b69b3521-a79f-44b0-99d8-9dc6d045f3c4","Type":"ContainerStarted","Data":"e5e1ddf28eacc5d338286c2b47c96f1b88b64157e6c4f3cfddc55e51a8724ffb"}
Sep 16 18:29:00.433082 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.432977    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:29:00.449129 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.449082    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d89597d46-nnpch" podStartSLOduration=1.449063115 podStartE2EDuration="1.449063115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:00.447771651 +0000 UTC m=+67.868258512" watchObservedRunningTime="2026-09-16 18:29:00.449063115 +0000 UTC m=+67.869549978"
Sep 16 18:29:00.560668 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560638    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560668 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560672    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9lrq\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560719    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560799    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.560882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.560870    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca\") pod \"cca8f41f-849f-4c4d-aafb-1b097deabf52\" (UID: \"cca8f41f-849f-4c4d-aafb-1b097deabf52\") "
Sep 16 18:29:00.563074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.562670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:29:00.563074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.562884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:29:00.563074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.563044    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:29:00.563994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.563830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:29:00.563994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.563922    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:29:00.565669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.565635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:29:00.566264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.566237    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq" (OuterVolumeSpecName: "kube-api-access-m9lrq") pod "cca8f41f-849f-4c4d-aafb-1b097deabf52" (UID: "cca8f41f-849f-4c4d-aafb-1b097deabf52"). InnerVolumeSpecName "kube-api-access-m9lrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:29:00.661711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661680    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-bound-sa-token\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661706    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9lrq\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-kube-api-access-m9lrq\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661728    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-installation-pull-secrets\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661738    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cca8f41f-849f-4c4d-aafb-1b097deabf52-image-registry-private-configuration\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661748    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cca8f41f-849f-4c4d-aafb-1b097deabf52-ca-trust-extracted\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661755    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-certificates\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:00.661838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:00.661763    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cca8f41f-849f-4c4d-aafb-1b097deabf52-trusted-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:01.433127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.433066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m49ct" event={"ID":"80d732e2-c418-40e7-bd17-6258bf894253","Type":"ContainerStarted","Data":"7101c130e439526b5e36fdf1bc41a465f5ef7ec61604b1f89ed7c04d17c7dd38"}
Sep 16 18:29:01.433655 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.433596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcfdf658d-mlcc6"
Sep 16 18:29:01.454069 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.454010    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m49ct" podStartSLOduration=67.477493988 podStartE2EDuration="1m8.453979864s" totalImagesPullingTime="976.485876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:01.450281499 +0000 UTC m=+68.870768356" watchObservedRunningTime="2026-09-16 18:29:01.453979864 +0000 UTC m=+68.874466726"
Sep 16 18:29:01.485098 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.485065    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bcfdf658d-mlcc6"]
Sep 16 18:29:01.489917 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.489862    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bcfdf658d-mlcc6"]
Sep 16 18:29:01.569213 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:01.569179    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cca8f41f-849f-4c4d-aafb-1b097deabf52-registry-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:29:02.439723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:02.439689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrwf8" event={"ID":"b69b3521-a79f-44b0-99d8-9dc6d045f3c4","Type":"ContainerStarted","Data":"d83797bee9ccd2daa115bcfd5e7c7df4016b05d5bf65d4a716f5046c959d3945"}
Sep 16 18:29:02.440216 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:02.439786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrwf8" event={"ID":"b69b3521-a79f-44b0-99d8-9dc6d045f3c4","Type":"ContainerStarted","Data":"118a1f9a07c2ccb226cb16f25d56313a948d21e3702a2e0f961e061348563dbc"}
Sep 16 18:29:02.459451 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:02.459405    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lrwf8" podStartSLOduration=33.388621181 podStartE2EDuration="35.459388464s" totalImagesPullingTime="2.070767283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:02.459049674 +0000 UTC m=+69.879536535" watchObservedRunningTime="2026-09-16 18:29:02.459388464 +0000 UTC m=+69.879875324"
Sep 16 18:29:02.779617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:02.779538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:29:02.782513 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:02.782492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5c79871-1520-4581-8a74-d4c1e6ca07d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mlb96\" (UID: \"a5c79871-1520-4581-8a74-d4c1e6ca07d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:29:03.009742 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:03.009701    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca8f41f-849f-4c4d-aafb-1b097deabf52" path="/var/lib/kubelet/pods/cca8f41f-849f-4c4d-aafb-1b097deabf52/volumes"
Sep 16 18:29:03.065845 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:03.065775    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-crvdp\""
Sep 16 18:29:03.073876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:03.073848    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"
Sep 16 18:29:03.443582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:03.443555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:29:03.841616 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:03.841575    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96"]
Sep 16 18:29:03.844391 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:03.844356    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c79871_1520_4581_8a74_d4c1e6ca07d3.slice/crio-5c80249bd47684c6a5bbcf6d398a997ce7cf43c0b63e4fd9ed2d47f19bb0457b WatchSource:0}: Error finding container 5c80249bd47684c6a5bbcf6d398a997ce7cf43c0b63e4fd9ed2d47f19bb0457b: Status 404 returned error can't find the container with id 5c80249bd47684c6a5bbcf6d398a997ce7cf43c0b63e4fd9ed2d47f19bb0457b
Sep 16 18:29:04.447852 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:04.447819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5fns9" event={"ID":"7c049995-694f-4ace-b1a7-6ea9f6d047a7","Type":"ContainerStarted","Data":"5e542d8ab0e8392a20ebb4df223d51fc62f3824ddd713274ea63105e11c4b69c"}
Sep 16 18:29:04.449196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:04.449175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96" event={"ID":"a5c79871-1520-4581-8a74-d4c1e6ca07d3","Type":"ContainerStarted","Data":"5c80249bd47684c6a5bbcf6d398a997ce7cf43c0b63e4fd9ed2d47f19bb0457b"}
Sep 16 18:29:04.465796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:04.465750    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5fns9" podStartSLOduration=67.194053548 podStartE2EDuration="1m11.465736678s" totalImagesPullingTime="4.27168313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:04.464697935 +0000 UTC m=+71.885184797" watchObservedRunningTime="2026-09-16 18:29:04.465736678 +0000 UTC m=+71.886223539"
Sep 16 18:29:06.459001 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:06.458969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96" event={"ID":"a5c79871-1520-4581-8a74-d4c1e6ca07d3","Type":"ContainerStarted","Data":"302d85c23376171e296738aa0ab2019228b79fabaad580058873d9d720319b24"}
Sep 16 18:29:06.478848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:06.478746    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mlb96" podStartSLOduration=34.087551029 podStartE2EDuration="36.478729183s" totalImagesPullingTime="2.391178154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:06.478158647 +0000 UTC m=+73.898645507" watchObservedRunningTime="2026-09-16 18:29:06.478729183 +0000 UTC m=+73.899216043"
Sep 16 18:29:13.452011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:13.451978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lrwf8"
Sep 16 18:29:15.638773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.638634    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-bdjqk"]
Sep 16 18:29:15.641199 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.641177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:15.643705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.643684    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 18:29:15.643815 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.643778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 18:29:15.643882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.643817    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xq55x\""
Sep 16 18:29:15.650751 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.650732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-bdjqk"]
Sep 16 18:29:15.770029 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.770004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbcwn\" (UniqueName: \"kubernetes.io/projected/7be202e6-ef77-4898-9f28-8b5bc58d1e10-kube-api-access-bbcwn\") pod \"downloads-76f99584f4-bdjqk\" (UID: \"7be202e6-ef77-4898-9f28-8b5bc58d1e10\") " pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:15.870298 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.870275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbcwn\" (UniqueName: \"kubernetes.io/projected/7be202e6-ef77-4898-9f28-8b5bc58d1e10-kube-api-access-bbcwn\") pod \"downloads-76f99584f4-bdjqk\" (UID: \"7be202e6-ef77-4898-9f28-8b5bc58d1e10\") " pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:15.886385 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.886364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbcwn\" (UniqueName: \"kubernetes.io/projected/7be202e6-ef77-4898-9f28-8b5bc58d1e10-kube-api-access-bbcwn\") pod \"downloads-76f99584f4-bdjqk\" (UID: \"7be202e6-ef77-4898-9f28-8b5bc58d1e10\") " pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:15.950706 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:15.950684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:16.193352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.193323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"]
Sep 16 18:29:16.195248 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.195230    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.198168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.198137    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 18:29:16.198168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.198139    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-98jw8\""
Sep 16 18:29:16.198440 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.198419    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:29:16.199083 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.199066    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:29:16.212689 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.212669    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"]
Sep 16 18:29:16.234707 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.232327    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-bdjqk"]
Sep 16 18:29:16.238527 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:16.238496    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be202e6_ef77_4898_9f28_8b5bc58d1e10.slice/crio-487586f157765b53100f4c2d01cd84369df64a84b000e600c78fd0866b974863 WatchSource:0}: Error finding container 487586f157765b53100f4c2d01cd84369df64a84b000e600c78fd0866b974863: Status 404 returned error can't find the container with id 487586f157765b53100f4c2d01cd84369df64a84b000e600c78fd0866b974863
Sep 16 18:29:16.264985 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.264967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"]
Sep 16 18:29:16.266906 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.266892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.271217 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271203    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 18:29:16.271345 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271324    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:29:16.271620 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271603    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nn7ng\""
Sep 16 18:29:16.271808 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 18:29:16.271871 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgh2v\" (UniqueName: \"kubernetes.io/projected/83bcd5f1-d03e-4d66-bcc6-022302423c73-kube-api-access-xgh2v\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.271871 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.271978 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.271903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bcd5f1-d03e-4d66-bcc6-022302423c73-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.272044 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.272026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.292422 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.292398    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"]
Sep 16 18:29:16.300798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.300780    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w9hh8"]
Sep 16 18:29:16.302653 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.302639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.305899 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.305879    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:29:16.305997 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.305917    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:29:16.306261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.306245    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:29:16.306261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.306258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-22lf8\""
Sep 16 18:29:16.372900 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.372869    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.372915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-sys\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.372981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-textfile\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwrln\" (UniqueName: \"kubernetes.io/projected/6de8231e-5c24-47f2-8f6a-c93ffaf79435-kube-api-access-jwrln\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a625b437-cd99-45ab-bc91-e76a67aa7528-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-tls\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg2tg\" (UniqueName: \"kubernetes.io/projected/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-api-access-pg2tg\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bcd5f1-d03e-4d66-bcc6-022302423c73-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.373370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.373370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.373370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-wtmp\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-metrics-client-ca\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-accelerators-collector-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-root\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.373615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgh2v\" (UniqueName: \"kubernetes.io/projected/83bcd5f1-d03e-4d66-bcc6-022302423c73-kube-api-access-xgh2v\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.373615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.373857 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.373781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bcd5f1-d03e-4d66-bcc6-022302423c73-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.375771 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.375755    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.375913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.375896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83bcd5f1-d03e-4d66-bcc6-022302423c73-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.382681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.382664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgh2v\" (UniqueName: \"kubernetes.io/projected/83bcd5f1-d03e-4d66-bcc6-022302423c73-kube-api-access-xgh2v\") pod \"openshift-state-metrics-67dcf8c979-ljz5b\" (UID: \"83bcd5f1-d03e-4d66-bcc6-022302423c73\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.474994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.474921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.474994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.474977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-wtmp\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-metrics-client-ca\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-accelerators-collector-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:29:16.475055    2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-root\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:29:16.475109    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls podName:a625b437-cd99-45ab-bc91-e76a67aa7528 nodeName:}" failed. No retries permitted until 2026-09-16 18:29:16.975092067 +0000 UTC m=+84.395578905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-vl425" (UID: "a625b437-cd99-45ab-bc91-e76a67aa7528") : secret "kube-state-metrics-tls" not found
Sep 16 18:29:16.475257 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-wtmp\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-root\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-sys\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-textfile\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwrln\" (UniqueName: \"kubernetes.io/projected/6de8231e-5c24-47f2-8f6a-c93ffaf79435-kube-api-access-jwrln\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de8231e-5c24-47f2-8f6a-c93ffaf79435-sys\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a625b437-cd99-45ab-bc91-e76a67aa7528-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-tls\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.475659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg2tg\" (UniqueName: \"kubernetes.io/projected/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-api-access-pg2tg\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-textfile\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.475784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-accelerators-collector-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.476119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.476153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.476158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6de8231e-5c24-47f2-8f6a-c93ffaf79435-metrics-client-ca\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.476723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.476355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a625b437-cd99-45ab-bc91-e76a67aa7528-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.477568 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.477537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.477982 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.477967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.478041 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.478019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6de8231e-5c24-47f2-8f6a-c93ffaf79435-node-exporter-tls\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.484595 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.484571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg2tg\" (UniqueName: \"kubernetes.io/projected/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-api-access-pg2tg\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.484595 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.484591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwrln\" (UniqueName: \"kubernetes.io/projected/6de8231e-5c24-47f2-8f6a-c93ffaf79435-kube-api-access-jwrln\") pod \"node-exporter-w9hh8\" (UID: \"6de8231e-5c24-47f2-8f6a-c93ffaf79435\") " pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.495569 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.495547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-bdjqk" event={"ID":"7be202e6-ef77-4898-9f28-8b5bc58d1e10","Type":"ContainerStarted","Data":"487586f157765b53100f4c2d01cd84369df64a84b000e600c78fd0866b974863"}
Sep 16 18:29:16.505753 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.505739    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"
Sep 16 18:29:16.611543 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.611517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w9hh8"
Sep 16 18:29:16.620627 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:16.620601    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de8231e_5c24_47f2_8f6a_c93ffaf79435.slice/crio-bc97566be4900995929628d32e615c21724c0b01ae14210bd7d785ce28157020 WatchSource:0}: Error finding container bc97566be4900995929628d32e615c21724c0b01ae14210bd7d785ce28157020: Status 404 returned error can't find the container with id bc97566be4900995929628d32e615c21724c0b01ae14210bd7d785ce28157020
Sep 16 18:29:16.778836 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.778812    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b"]
Sep 16 18:29:16.780889 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:16.780867    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bcd5f1_d03e_4d66_bcc6_022302423c73.slice/crio-ee3de6e964fe7fc9a533156918b192bb4584584230f3e0fe091b40acdc915c97 WatchSource:0}: Error finding container ee3de6e964fe7fc9a533156918b192bb4584584230f3e0fe091b40acdc915c97: Status 404 returned error can't find the container with id ee3de6e964fe7fc9a533156918b192bb4584584230f3e0fe091b40acdc915c97
Sep 16 18:29:16.980001 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.979968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:16.982696 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:16.982672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a625b437-cd99-45ab-bc91-e76a67aa7528-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-vl425\" (UID: \"a625b437-cd99-45ab-bc91-e76a67aa7528\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:17.175681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.175612    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"
Sep 16 18:29:17.302396 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.302361    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:29:17.308982 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.308958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.311191    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.311553    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.311724    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.311909    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.312124    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.313176    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bkvkh\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.313364    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.313528    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 18:29:17.319516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.313685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 18:29:17.323168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.322997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:29:17.323804 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.323418    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 18:29:17.383179 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383330 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383330 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383330 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383330 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383330 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383346    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwk4l\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.383582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.383546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.474124 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.473961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-vl425"]
Sep 16 18:29:17.480154 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:17.480078    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda625b437_cd99_45ab_bc91_e76a67aa7528.slice/crio-672c9e7697cd3756b3869efa0f2a2eba5fc155fa360d4cd52b7071490d18089e WatchSource:0}: Error finding container 672c9e7697cd3756b3869efa0f2a2eba5fc155fa360d4cd52b7071490d18089e: Status 404 returned error can't find the container with id 672c9e7697cd3756b3869efa0f2a2eba5fc155fa360d4cd52b7071490d18089e
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.483865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.483905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.483956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.483981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwk4l\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.485669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.484258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.487654 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.487628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.488862 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.488836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.490202 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.489697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.494162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.493699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.494162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.493891    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.495165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.494587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.495165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.494870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.495165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.495116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.495165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.495133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.495950 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.495803    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.497169 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.497144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.499889 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.499844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.504164 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.504141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwk4l\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l\") pod \"alertmanager-main-0\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:17.505237 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.505208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w9hh8" event={"ID":"6de8231e-5c24-47f2-8f6a-c93ffaf79435","Type":"ContainerStarted","Data":"bc97566be4900995929628d32e615c21724c0b01ae14210bd7d785ce28157020"}
Sep 16 18:29:17.508137 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.508095    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b" event={"ID":"83bcd5f1-d03e-4d66-bcc6-022302423c73","Type":"ContainerStarted","Data":"ce3da04f49f568315a6ec823ccbbbeaf1c07506b8202314f32314a250acbdd33"}
Sep 16 18:29:17.508228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.508164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b" event={"ID":"83bcd5f1-d03e-4d66-bcc6-022302423c73","Type":"ContainerStarted","Data":"472ccb7556c29755623e44dc6b5158ae4d4e95b1e2860b7e666c974faa12ab64"}
Sep 16 18:29:17.508228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.508188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b" event={"ID":"83bcd5f1-d03e-4d66-bcc6-022302423c73","Type":"ContainerStarted","Data":"ee3de6e964fe7fc9a533156918b192bb4584584230f3e0fe091b40acdc915c97"}
Sep 16 18:29:17.509711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.509686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425" event={"ID":"a625b437-cd99-45ab-bc91-e76a67aa7528","Type":"ContainerStarted","Data":"672c9e7697cd3756b3869efa0f2a2eba5fc155fa360d4cd52b7071490d18089e"}
Sep 16 18:29:17.627291 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:17.627166    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:29:18.208703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.208677    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d8976999f-7b98l"]
Sep 16 18:29:18.211724 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.211703    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.214717 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.214481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 16 18:29:18.215533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215142    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6oc3i15pk7bpl\""
Sep 16 18:29:18.215533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 16 18:29:18.215533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215270    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 16 18:29:18.215533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215286    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-75zpc\""
Sep 16 18:29:18.215533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215394    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 16 18:29:18.215787 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.215542    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 16 18:29:18.227049 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.226987    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d8976999f-7b98l"]
Sep 16 18:29:18.295529 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-grpc-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295778 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295675    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295778 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gp9t\" (UniqueName: \"kubernetes.io/projected/0482fc37-0610-4cba-9562-4fb6d8c356c1-kube-api-access-8gp9t\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295778 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0482fc37-0610-4cba-9562-4fb6d8c356c1-metrics-client-ca\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.295924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.295860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.353238 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.353085    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:29:18.355897 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:18.355864    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822f2d94_8441_4f40_b6bd_9f9e4888cf75.slice/crio-c37bebb968e088244269b1a2973377632c8a52691f0d502306ea175631f064bf WatchSource:0}: Error finding container c37bebb968e088244269b1a2973377632c8a52691f0d502306ea175631f064bf: Status 404 returned error can't find the container with id c37bebb968e088244269b1a2973377632c8a52691f0d502306ea175631f064bf
Sep 16 18:29:18.396597 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-grpc-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gp9t\" (UniqueName: \"kubernetes.io/projected/0482fc37-0610-4cba-9562-4fb6d8c356c1-kube-api-access-8gp9t\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0482fc37-0610-4cba-9562-4fb6d8c356c1-metrics-client-ca\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.396882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.396826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.398095 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.398039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0482fc37-0610-4cba-9562-4fb6d8c356c1-metrics-client-ca\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.400271 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.400249    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.400504 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.400478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.400747 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.400725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.401351 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.401231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.401351 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.401313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-grpc-tls\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.403157 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.403112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0482fc37-0610-4cba-9562-4fb6d8c356c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.406626 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.406604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gp9t\" (UniqueName: \"kubernetes.io/projected/0482fc37-0610-4cba-9562-4fb6d8c356c1-kube-api-access-8gp9t\") pod \"thanos-querier-5d8976999f-7b98l\" (UID: \"0482fc37-0610-4cba-9562-4fb6d8c356c1\") " pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.516470 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.516382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"c37bebb968e088244269b1a2973377632c8a52691f0d502306ea175631f064bf"}
Sep 16 18:29:18.518640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.518601    2576 generic.go:356] "Generic (PLEG): container finished" podID="6de8231e-5c24-47f2-8f6a-c93ffaf79435" containerID="24021f24c1433285086b7e63c209356cd73c7d5388578f6dff1a242715c6e930" exitCode=0
Sep 16 18:29:18.518736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.518669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w9hh8" event={"ID":"6de8231e-5c24-47f2-8f6a-c93ffaf79435","Type":"ContainerDied","Data":"24021f24c1433285086b7e63c209356cd73c7d5388578f6dff1a242715c6e930"}
Sep 16 18:29:18.521490 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.521460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b" event={"ID":"83bcd5f1-d03e-4d66-bcc6-022302423c73","Type":"ContainerStarted","Data":"f3682359e5797f61fc5133411c40a7bc21d92c1e830167a1eb928c964715ddbc"}
Sep 16 18:29:18.525034 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.524335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:18.558732 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.558656    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-ljz5b" podStartSLOduration=1.389898186 podStartE2EDuration="2.558639869s" totalImagesPullingTime="1.168741683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:18.558093291 +0000 UTC m=+85.978580154" watchObservedRunningTime="2026-09-16 18:29:18.558639869 +0000 UTC m=+85.979126729"
Sep 16 18:29:18.819967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:18.819922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d8976999f-7b98l"]
Sep 16 18:29:19.046713 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:19.046681    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0482fc37_0610_4cba_9562_4fb6d8c356c1.slice/crio-2ae8f451e385b3eda73c3f4289499e8bc769f099a00698c62300ce1f249692c9 WatchSource:0}: Error finding container 2ae8f451e385b3eda73c3f4289499e8bc769f099a00698c62300ce1f249692c9: Status 404 returned error can't find the container with id 2ae8f451e385b3eda73c3f4289499e8bc769f099a00698c62300ce1f249692c9
Sep 16 18:29:19.530270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.530208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425" event={"ID":"a625b437-cd99-45ab-bc91-e76a67aa7528","Type":"ContainerStarted","Data":"61caa9207d8668ff583f20c24dde60af7ffc984d2cb4830af90ee2b05749f572"}
Sep 16 18:29:19.530714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.530343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425" event={"ID":"a625b437-cd99-45ab-bc91-e76a67aa7528","Type":"ContainerStarted","Data":"4275c445bdcb8aca5559035b5dd726574737c55c8230c3ef852bc55adb4e2c03"}
Sep 16 18:29:19.530714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.530374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425" event={"ID":"a625b437-cd99-45ab-bc91-e76a67aa7528","Type":"ContainerStarted","Data":"8f7b3f4f9fd7e18281b888f46a4f4b9a219bfccf5547056f8d6d97c558cdf5ab"}
Sep 16 18:29:19.533739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.533650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w9hh8" event={"ID":"6de8231e-5c24-47f2-8f6a-c93ffaf79435","Type":"ContainerStarted","Data":"98b83d0f6699c388ede4bb7bb929912cf41dd34e528d2117266c40208e944535"}
Sep 16 18:29:19.533739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.533705    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w9hh8" event={"ID":"6de8231e-5c24-47f2-8f6a-c93ffaf79435","Type":"ContainerStarted","Data":"daf6c3f983da92fb17f12893d4eabb156a2b314294a48355a4cd99d6990bb985"}
Sep 16 18:29:19.535488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.535461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"2ae8f451e385b3eda73c3f4289499e8bc769f099a00698c62300ce1f249692c9"}
Sep 16 18:29:19.553675 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.553152    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-vl425" podStartSLOduration=1.965027059 podStartE2EDuration="3.553134198s" totalImagesPullingTime="1.588107139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:19.551830957 +0000 UTC m=+86.972317820" watchObservedRunningTime="2026-09-16 18:29:19.553134198 +0000 UTC m=+86.973621060"
Sep 16 18:29:19.880429 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.880348    2576 patch_prober.go:28] interesting pod/image-registry-6d89597d46-nnpch container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:29:19.880636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:19.880416    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d89597d46-nnpch" podUID="6279a734-dcbb-4640-9912-60eeddc3144f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:29:20.542049 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.542009    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774" exitCode=0
Sep 16 18:29:20.542506 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.542074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"}
Sep 16 18:29:20.574532 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.574484    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w9hh8" podStartSLOduration=3.041256989 podStartE2EDuration="4.574462492s" totalImagesPullingTime="1.451472794s" totalInitContainerRuntime="81.732709ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:19.574874184 +0000 UTC m=+86.995361044" watchObservedRunningTime="2026-09-16 18:29:20.574462492 +0000 UTC m=+87.994949354"
Sep 16 18:29:20.973418 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.973304    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"]
Sep 16 18:29:20.975757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.975729    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:20.978212 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.978190    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4bkxm\""
Sep 16 18:29:20.978333 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.978233    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 16 18:29:20.986294 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:20.986270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"]
Sep 16 18:29:21.122732 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.122695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.122902 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.122752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5zc\" (UniqueName: \"kubernetes.io/projected/45fee104-8448-41a8-a471-7bd19f193c58-kube-api-access-2p5zc\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.223581 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.223240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.223581 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.223291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5zc\" (UniqueName: \"kubernetes.io/projected/45fee104-8448-41a8-a471-7bd19f193c58-kube-api-access-2p5zc\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.223581 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:29:21.223370    2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 16 18:29:21.223581 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:29:21.223436    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert podName:45fee104-8448-41a8-a471-7bd19f193c58 nodeName:}" failed. No retries permitted until 2026-09-16 18:29:21.723418975 +0000 UTC m=+89.143905816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-k68qr" (UID: "45fee104-8448-41a8-a471-7bd19f193c58") : secret "monitoring-plugin-cert" not found
Sep 16 18:29:21.237823 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.237788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5zc\" (UniqueName: \"kubernetes.io/projected/45fee104-8448-41a8-a471-7bd19f193c58-kube-api-access-2p5zc\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.439662 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.439621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d89597d46-nnpch"
Sep 16 18:29:21.464987 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.464724    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-747c5b4957-np4rj"]
Sep 16 18:29:21.471413 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.471386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.474314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.474258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 18:29:21.474482 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.474312    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 18:29:21.474611 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.474354    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 18:29:21.475194 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.475175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tjbrr\""
Sep 16 18:29:21.475363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.475324    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 18:29:21.475586 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.475546    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 18:29:21.484848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.484826    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 18:29:21.486244 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.486223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747c5b4957-np4rj"]
Sep 16 18:29:21.627205 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-metrics-client-ca\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-federate-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvnrq\" (UniqueName: \"kubernetes.io/projected/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-kube-api-access-dvnrq\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-serving-certs-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.627907 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.627766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-serving-certs-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729165 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-metrics-client-ca\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729263 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.729263 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-federate-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729263 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvnrq\" (UniqueName: \"kubernetes.io/projected/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-kube-api-access-dvnrq\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.729395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.729346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.731663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.731290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.731663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.731438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-serving-certs-ca-bundle\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.731663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.731643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-metrics-client-ca\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.733276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.733241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/45fee104-8448-41a8-a471-7bd19f193c58-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k68qr\" (UID: \"45fee104-8448-41a8-a471-7bd19f193c58\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:21.733361 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.733325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-telemeter-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.734738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.734711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.736138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.736096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-federate-client-tls\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.736319 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.736301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-secret-telemeter-client\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.742729 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.742687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvnrq\" (UniqueName: \"kubernetes.io/projected/6f91dae3-62e0-4cb3-9019-ffe0ceb3675e-kube-api-access-dvnrq\") pod \"telemeter-client-747c5b4957-np4rj\" (UID: \"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e\") " pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.787621 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.787594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj"
Sep 16 18:29:21.887703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:21.887674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:22.078396 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.078371    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747c5b4957-np4rj"]
Sep 16 18:29:22.080064 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:22.080039    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f91dae3_62e0_4cb3_9019_ffe0ceb3675e.slice/crio-61af3cf781e2e73b8be9afa6374c57bce518d5bb711d804621d97e387ebec617 WatchSource:0}: Error finding container 61af3cf781e2e73b8be9afa6374c57bce518d5bb711d804621d97e387ebec617: Status 404 returned error can't find the container with id 61af3cf781e2e73b8be9afa6374c57bce518d5bb711d804621d97e387ebec617
Sep 16 18:29:22.166049 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.166022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"]
Sep 16 18:29:22.170757 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:22.170731    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fee104_8448_41a8_a471_7bd19f193c58.slice/crio-4e76dfe48d1b57690edd7530210d25b638a373f5c435791e35c0ae938a6a1c10 WatchSource:0}: Error finding container 4e76dfe48d1b57690edd7530210d25b638a373f5c435791e35c0ae938a6a1c10: Status 404 returned error can't find the container with id 4e76dfe48d1b57690edd7530210d25b638a373f5c435791e35c0ae938a6a1c10
Sep 16 18:29:22.556833 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.556753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr" event={"ID":"45fee104-8448-41a8-a471-7bd19f193c58","Type":"ContainerStarted","Data":"4e76dfe48d1b57690edd7530210d25b638a373f5c435791e35c0ae938a6a1c10"}
Sep 16 18:29:22.558169 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.558140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj" event={"ID":"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e","Type":"ContainerStarted","Data":"61af3cf781e2e73b8be9afa6374c57bce518d5bb711d804621d97e387ebec617"}
Sep 16 18:29:22.561085 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.561048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"fc8126276373b13f365868cccb40174fea34b7823db15e68a5c626e0c8ee62c0"}
Sep 16 18:29:22.561159 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.561122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"d8c35f2914bb9dc993f30e4d6741577ebe812629a1a193429b68b0c2c03f28cd"}
Sep 16 18:29:22.561159 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:22.561155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"68e4a79444d37a0fcdebaf0ead3c777923e69c13130cf17ca8da31c91fb1a915"}
Sep 16 18:29:23.573745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573545    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"}
Sep 16 18:29:23.573745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"}
Sep 16 18:29:23.573745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573667    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"}
Sep 16 18:29:23.573745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"}
Sep 16 18:29:23.573745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"}
Sep 16 18:29:23.574292 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.573775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerStarted","Data":"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"}
Sep 16 18:29:23.577855 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.577826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"3b9007ab11a08e6775825c9a75e3ec2b912c4615eafd87d7650e0ee786c4a212"}
Sep 16 18:29:23.577998 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.577905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"17134bc2f7d1a030add1f6acfafa05c6e8bc138d9713940a25698f4e12dd76c9"}
Sep 16 18:29:23.577998 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.577968    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" event={"ID":"0482fc37-0610-4cba-9562-4fb6d8c356c1","Type":"ContainerStarted","Data":"3287ca9028dd12aa607b3e0991f1106b079952f91f1aa57f0203a798cc358a46"}
Sep 16 18:29:23.578815 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.578789    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:23.606690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.606448    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.715791792 podStartE2EDuration="6.60642731s" totalImagesPullingTime="3.868552052s" totalInitContainerRuntime="22.083466ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:23.605101082 +0000 UTC m=+91.025587943" watchObservedRunningTime="2026-09-16 18:29:23.60642731 +0000 UTC m=+91.026914148"
Sep 16 18:29:23.629450 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:23.628633    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l" podStartSLOduration=2.018735641 podStartE2EDuration="5.628612849s" totalImagesPullingTime="3.609877208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:23.627765517 +0000 UTC m=+91.048252382" watchObservedRunningTime="2026-09-16 18:29:23.628612849 +0000 UTC m=+91.049099711"
Sep 16 18:29:24.588901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:24.588789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr" event={"ID":"45fee104-8448-41a8-a471-7bd19f193c58","Type":"ContainerStarted","Data":"8bf187213a94a228a9436b29be1dade9e214dfd0d38a41f3353493d3a4fe0b01"}
Sep 16 18:29:24.589304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:24.589251    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:24.596223 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:24.596199    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr"
Sep 16 18:29:24.609676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:24.609627    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k68qr" podStartSLOduration=2.733751748 podStartE2EDuration="4.60960902s" totalImagesPullingTime="1.875857272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:24.608983371 +0000 UTC m=+92.029470235" watchObservedRunningTime="2026-09-16 18:29:24.60960902 +0000 UTC m=+92.030095888"
Sep 16 18:29:25.595727 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.595688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj" event={"ID":"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e","Type":"ContainerStarted","Data":"1f6a21d698dee7776e49d12a83394f503a9c89d0b9cee311b03ba10fb4bf0950"}
Sep 16 18:29:25.596206 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.595766    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj" event={"ID":"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e","Type":"ContainerStarted","Data":"c2abf66ce8559c2afacbbf966f49c2e0b6b78f97ec059a9bd739000c239230c6"}
Sep 16 18:29:25.596206 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.595795    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj" event={"ID":"6f91dae3-62e0-4cb3-9019-ffe0ceb3675e","Type":"ContainerStarted","Data":"44731e82385b5568a8d0061e709f5a89c59a94cb8902a7ee762a74d3c172b5a2"}
Sep 16 18:29:25.624543 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.624489    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-747c5b4957-np4rj" podStartSLOduration=1.896118851 podStartE2EDuration="4.624467919s" totalImagesPullingTime="2.728349068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:25.622367565 +0000 UTC m=+93.042854437" watchObservedRunningTime="2026-09-16 18:29:25.624467919 +0000 UTC m=+93.044954782"
Sep 16 18:29:25.922617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.922585    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:29:25.926032 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.926004    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:25.929226 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.929201    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 18:29:25.929328 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.929285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9nrmt\""
Sep 16 18:29:25.929458 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.929441    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 18:29:25.930136 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.930113    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 18:29:25.930273 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.930168    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 18:29:25.930273 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.930113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 18:29:25.936197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:25.936174    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:29:26.071218 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.071354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071229    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.071354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.071458 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.071458 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4kc4\" (UniqueName: \"kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.071541 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.071481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172253 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172253 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4kc4\" (UniqueName: \"kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172253 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172253 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.172253 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.173019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.173019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.172970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.173019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.173002    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.174799 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.174776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.175271 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.175250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.184284 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.184261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4kc4\" (UniqueName: \"kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4\") pod \"console-6d956ff596-zbx4c\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") " pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.236995 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.236972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:26.517762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.517739    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:29:26.520512 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:26.520482    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374df99c_32ba_49d2_a155_897d231deac1.slice/crio-d7f037917295c406fd4eafb68368bc08c6c594729e58685fb57d3dac89550d1b WatchSource:0}: Error finding container d7f037917295c406fd4eafb68368bc08c6c594729e58685fb57d3dac89550d1b: Status 404 returned error can't find the container with id d7f037917295c406fd4eafb68368bc08c6c594729e58685fb57d3dac89550d1b
Sep 16 18:29:26.601979 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:26.601919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d956ff596-zbx4c" event={"ID":"374df99c-32ba-49d2-a155-897d231deac1","Type":"ContainerStarted","Data":"d7f037917295c406fd4eafb68368bc08c6c594729e58685fb57d3dac89550d1b"}
Sep 16 18:29:30.429574 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:30.429547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fcbbv"
Sep 16 18:29:30.602579 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:30.602548    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d8976999f-7b98l"
Sep 16 18:29:34.371725 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.371695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:29:34.389949 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.389905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:29:34.390091 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.390052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.399050 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.398416    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 18:29:34.445710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt5nc\" (UniqueName: \"kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.445919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.445858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546385 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546682 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt5nc\" (UniqueName: \"kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546682 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.546682 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.546616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.547393 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.547366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.547513 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.547475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.547711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.547663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.547963 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.547899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.549508 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.549483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.549619 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.549483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.556342 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.556317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt5nc\" (UniqueName: \"kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc\") pod \"console-6cc9f786f5-w6fpk\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") " pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:34.643377 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.643338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-bdjqk" event={"ID":"7be202e6-ef77-4898-9f28-8b5bc58d1e10","Type":"ContainerStarted","Data":"18a78e22c6ac8a8fb9a3dba99dbba55019cdabd1ef248a8f685db0e517ae77ae"}
Sep 16 18:29:34.643796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.643701    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:34.645227 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.645198    2576 patch_prober.go:28] interesting pod/downloads-76f99584f4-bdjqk container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" start-of-body=
Sep 16 18:29:34.645531 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.645330    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-bdjqk" podUID="7be202e6-ef77-4898-9f28-8b5bc58d1e10" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused"
Sep 16 18:29:34.662003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.661597    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-bdjqk" podStartSLOduration=1.710023311 podStartE2EDuration="19.661581566s" totalImagesPullingTime="17.951558255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:34.660746077 +0000 UTC m=+102.081232938" watchObservedRunningTime="2026-09-16 18:29:34.661581566 +0000 UTC m=+102.082068464"
Sep 16 18:29:34.702632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:34.702571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:35.026440 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:35.025741    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:29:35.032303 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:29:35.032267    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ca2527_05d8_40c4_b349_6c946ba858ed.slice/crio-ec4697c4334c3b70606347159b08a3ca9b626831c530860a6cea52e310044ab2 WatchSource:0}: Error finding container ec4697c4334c3b70606347159b08a3ca9b626831c530860a6cea52e310044ab2: Status 404 returned error can't find the container with id ec4697c4334c3b70606347159b08a3ca9b626831c530860a6cea52e310044ab2
Sep 16 18:29:35.651075 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:35.650767    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc9f786f5-w6fpk" event={"ID":"79ca2527-05d8-40c4-b349-6c946ba858ed","Type":"ContainerStarted","Data":"ec4697c4334c3b70606347159b08a3ca9b626831c530860a6cea52e310044ab2"}
Sep 16 18:29:35.664483 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:35.664263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-bdjqk"
Sep 16 18:29:38.672442 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:38.672388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d956ff596-zbx4c" event={"ID":"374df99c-32ba-49d2-a155-897d231deac1","Type":"ContainerStarted","Data":"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"}
Sep 16 18:29:38.674441 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:38.674406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc9f786f5-w6fpk" event={"ID":"79ca2527-05d8-40c4-b349-6c946ba858ed","Type":"ContainerStarted","Data":"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"}
Sep 16 18:29:38.691164 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:38.691113    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d956ff596-zbx4c" podStartSLOduration=2.272303809 podStartE2EDuration="13.691093806s" totalImagesPullingTime="11.418789997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:38.689222882 +0000 UTC m=+106.109709743" watchObservedRunningTime="2026-09-16 18:29:38.691093806 +0000 UTC m=+106.111580666"
Sep 16 18:29:38.706738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:38.706313    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cc9f786f5-w6fpk" podStartSLOduration=1.669888998 podStartE2EDuration="4.706295252s" totalImagesPullingTime="3.036406254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:38.706071376 +0000 UTC m=+106.126558236" watchObservedRunningTime="2026-09-16 18:29:38.706295252 +0000 UTC m=+106.126782114"
Sep 16 18:29:44.703110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:44.703075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:44.703560 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:44.703163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:44.709563 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:44.709519    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:44.715605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:44.715583    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:29:44.779321 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:44.779293    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:29:46.237763 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:46.237730    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:29:52.742983 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:52.742946    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef541df6-cc2e-4d36-b932-392a6411733b" containerID="43061decc38190f39188cb60cae55369e76b100d999e7a5698dbb48cdd145d5c" exitCode=0
Sep 16 18:29:52.743431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:52.743000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-czsgb" event={"ID":"ef541df6-cc2e-4d36-b932-392a6411733b","Type":"ContainerDied","Data":"43061decc38190f39188cb60cae55369e76b100d999e7a5698dbb48cdd145d5c"}
Sep 16 18:29:52.743431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:52.743288    2576 scope.go:117] "RemoveContainer" containerID="43061decc38190f39188cb60cae55369e76b100d999e7a5698dbb48cdd145d5c"
Sep 16 18:29:53.750708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:29:53.750673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-czsgb" event={"ID":"ef541df6-cc2e-4d36-b932-392a6411733b","Type":"ContainerStarted","Data":"f25c04ca06b74779ee01c9712c92fbb4255d5099dfcf3717b0aac66e4bd762c1"}
Sep 16 18:30:09.798785 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:09.798729    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d956ff596-zbx4c" podUID="374df99c-32ba-49d2-a155-897d231deac1" containerName="console" containerID="cri-o://55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3" gracePeriod=15
Sep 16 18:30:10.070014 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.069952    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:30:10.149274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4kc4\" (UniqueName: \"kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149303    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149341    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149619 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149546    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149630    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert\") pod \"374df99c-32ba-49d2-a155-897d231deac1\" (UID: \"374df99c-32ba-49d2-a155-897d231deac1\") "
Sep 16 18:30:10.149749 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149708    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config" (OuterVolumeSpecName: "console-config") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:30:10.149873 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149724    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca" (OuterVolumeSpecName: "service-ca") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:30:10.150120 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149891    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:30:10.150120 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149925    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.150120 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.149978    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.151703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.151679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:10.151785 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.151735    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4" (OuterVolumeSpecName: "kube-api-access-c4kc4") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "kube-api-access-c4kc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:30:10.151785 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.151776    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "374df99c-32ba-49d2-a155-897d231deac1" (UID: "374df99c-32ba-49d2-a155-897d231deac1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:10.251008 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.250975    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.251008 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.251002    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4kc4\" (UniqueName: \"kubernetes.io/projected/374df99c-32ba-49d2-a155-897d231deac1-kube-api-access-c4kc4\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.251008 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.251012    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/374df99c-32ba-49d2-a155-897d231deac1-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.251207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.251020    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/374df99c-32ba-49d2-a155-897d231deac1-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:10.832557 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.832521    2576 generic.go:356] "Generic (PLEG): container finished" podID="374df99c-32ba-49d2-a155-897d231deac1" containerID="55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3" exitCode=0
Sep 16 18:30:10.832954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.832563    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d956ff596-zbx4c"
Sep 16 18:30:10.832954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.832609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d956ff596-zbx4c" event={"ID":"374df99c-32ba-49d2-a155-897d231deac1","Type":"ContainerDied","Data":"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"}
Sep 16 18:30:10.832954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.832683    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d956ff596-zbx4c" event={"ID":"374df99c-32ba-49d2-a155-897d231deac1","Type":"ContainerDied","Data":"d7f037917295c406fd4eafb68368bc08c6c594729e58685fb57d3dac89550d1b"}
Sep 16 18:30:10.832954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.832705    2576 scope.go:117] "RemoveContainer" containerID="55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"
Sep 16 18:30:10.841916 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.841900    2576 scope.go:117] "RemoveContainer" containerID="55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"
Sep 16 18:30:10.842210 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:10.842194    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3\": container with ID starting with 55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3 not found: ID does not exist" containerID="55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"
Sep 16 18:30:10.842256 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.842220    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3"} err="failed to get container status \"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3\": rpc error: code = NotFound desc = could not find container \"55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3\": container with ID starting with 55f5c534031120cea94a8b6513e282fa979d41693a00ab95ba18934effe551e3 not found: ID does not exist"
Sep 16 18:30:10.855079 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.855057    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:30:10.860950 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:10.860915    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d956ff596-zbx4c"]
Sep 16 18:30:11.010535 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:11.010495    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374df99c-32ba-49d2-a155-897d231deac1" path="/var/lib/kubelet/pods/374df99c-32ba-49d2-a155-897d231deac1/volumes"
Sep 16 18:30:36.688316 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688286    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:36.688802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688664    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="alertmanager" containerID="cri-o://7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab" gracePeriod=120
Sep 16 18:30:36.688802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688728    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-web" containerID="cri-o://46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a" gracePeriod=120
Sep 16 18:30:36.688802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688767    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="config-reloader" containerID="cri-o://ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7" gracePeriod=120
Sep 16 18:30:36.688999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688804    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="prom-label-proxy" containerID="cri-o://b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db" gracePeriod=120
Sep 16 18:30:36.688999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688763    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy" containerID="cri-o://9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305" gracePeriod=120
Sep 16 18:30:36.688999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.688728    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-metric" containerID="cri-o://45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f" gracePeriod=120
Sep 16 18:30:36.944572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944514    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db" exitCode=0
Sep 16 18:30:36.944572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944537    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305" exitCode=0
Sep 16 18:30:36.944572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944544    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7" exitCode=0
Sep 16 18:30:36.944572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944549    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab" exitCode=0
Sep 16 18:30:36.944839 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"}
Sep 16 18:30:36.944839 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"}
Sep 16 18:30:36.944839 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"}
Sep 16 18:30:36.944839 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:36.944765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"}
Sep 16 18:30:37.920172 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.920151    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:37.951667 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951644    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f" exitCode=0
Sep 16 18:30:37.951667 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951664    2576 generic.go:356] "Generic (PLEG): container finished" podID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerID="46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a" exitCode=0
Sep 16 18:30:37.951806 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951715    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:37.951806 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951729    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"}
Sep 16 18:30:37.951806 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"}
Sep 16 18:30:37.951893 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"822f2d94-8441-4f40-b6bd-9f9e4888cf75","Type":"ContainerDied","Data":"c37bebb968e088244269b1a2973377632c8a52691f0d502306ea175631f064bf"}
Sep 16 18:30:37.951893 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.951868    2576 scope.go:117] "RemoveContainer" containerID="b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"
Sep 16 18:30:37.954015 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.953997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954139 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954031    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954139 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954062    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954139 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954092    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954139 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954143    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954196    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwk4l\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954241    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954328    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954679 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954380    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.954679 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954408    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume\") pod \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\" (UID: \"822f2d94-8441-4f40-b6bd-9f9e4888cf75\") "
Sep 16 18:30:37.956051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.954957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:30:37.956051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.955762    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:30:37.956051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.956002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:30:37.957699 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.957671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume" (OuterVolumeSpecName: "config-volume") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.957798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.957711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:30:37.958218 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.958144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.959099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.959047    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.959448 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.959422    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.959873 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.959847    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out" (OuterVolumeSpecName: "config-out") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:30:37.960068 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.960041    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.960856 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.960618    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l" (OuterVolumeSpecName: "kube-api-access-qwk4l") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "kube-api-access-qwk4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:30:37.965088 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.964963    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.972858 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.972817    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config" (OuterVolumeSpecName: "web-config") pod "822f2d94-8441-4f40-b6bd-9f9e4888cf75" (UID: "822f2d94-8441-4f40-b6bd-9f9e4888cf75"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:37.973594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.973572    2576 scope.go:117] "RemoveContainer" containerID="45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"
Sep 16 18:30:37.982360 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.982337    2576 scope.go:117] "RemoveContainer" containerID="9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"
Sep 16 18:30:37.989028 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.989008    2576 scope.go:117] "RemoveContainer" containerID="46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"
Sep 16 18:30:37.996617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:37.996600    2576 scope.go:117] "RemoveContainer" containerID="ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"
Sep 16 18:30:38.003740 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.003721    2576 scope.go:117] "RemoveContainer" containerID="7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"
Sep 16 18:30:38.010762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.010746    2576 scope.go:117] "RemoveContainer" containerID="6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"
Sep 16 18:30:38.017558 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.017532    2576 scope.go:117] "RemoveContainer" containerID="b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"
Sep 16 18:30:38.017835 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.017813    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db\": container with ID starting with b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db not found: ID does not exist" containerID="b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"
Sep 16 18:30:38.017887 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.017848    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"} err="failed to get container status \"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db\": rpc error: code = NotFound desc = could not find container \"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db\": container with ID starting with b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db not found: ID does not exist"
Sep 16 18:30:38.017887 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.017874    2576 scope.go:117] "RemoveContainer" containerID="45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"
Sep 16 18:30:38.018137 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.018116    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f\": container with ID starting with 45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f not found: ID does not exist" containerID="45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"
Sep 16 18:30:38.018215 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018146    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"} err="failed to get container status \"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f\": rpc error: code = NotFound desc = could not find container \"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f\": container with ID starting with 45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f not found: ID does not exist"
Sep 16 18:30:38.018215 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018167    2576 scope.go:117] "RemoveContainer" containerID="9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"
Sep 16 18:30:38.018443 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.018420    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305\": container with ID starting with 9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305 not found: ID does not exist" containerID="9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"
Sep 16 18:30:38.018490 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018457    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"} err="failed to get container status \"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305\": rpc error: code = NotFound desc = could not find container \"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305\": container with ID starting with 9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305 not found: ID does not exist"
Sep 16 18:30:38.018490 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018476    2576 scope.go:117] "RemoveContainer" containerID="46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"
Sep 16 18:30:38.018733 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.018715    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a\": container with ID starting with 46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a not found: ID does not exist" containerID="46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"
Sep 16 18:30:38.018782 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018741    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"} err="failed to get container status \"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a\": rpc error: code = NotFound desc = could not find container \"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a\": container with ID starting with 46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a not found: ID does not exist"
Sep 16 18:30:38.018782 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.018758    2576 scope.go:117] "RemoveContainer" containerID="ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"
Sep 16 18:30:38.019023 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.019007    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7\": container with ID starting with ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7 not found: ID does not exist" containerID="ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"
Sep 16 18:30:38.019069 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019026    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"} err="failed to get container status \"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7\": rpc error: code = NotFound desc = could not find container \"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7\": container with ID starting with ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7 not found: ID does not exist"
Sep 16 18:30:38.019069 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019040    2576 scope.go:117] "RemoveContainer" containerID="7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"
Sep 16 18:30:38.019306 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.019273    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab\": container with ID starting with 7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab not found: ID does not exist" containerID="7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"
Sep 16 18:30:38.019376 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019315    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"} err="failed to get container status \"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab\": rpc error: code = NotFound desc = could not find container \"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab\": container with ID starting with 7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab not found: ID does not exist"
Sep 16 18:30:38.019376 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019336    2576 scope.go:117] "RemoveContainer" containerID="6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"
Sep 16 18:30:38.019588 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:30:38.019572    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774\": container with ID starting with 6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774 not found: ID does not exist" containerID="6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"
Sep 16 18:30:38.019654 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019592    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"} err="failed to get container status \"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774\": rpc error: code = NotFound desc = could not find container \"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774\": container with ID starting with 6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774 not found: ID does not exist"
Sep 16 18:30:38.019654 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019606    2576 scope.go:117] "RemoveContainer" containerID="b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"
Sep 16 18:30:38.019844 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019828    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db"} err="failed to get container status \"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db\": rpc error: code = NotFound desc = could not find container \"b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db\": container with ID starting with b463b88e6358f7fe905f8546be5a9a0c6786c76228f6b8a3b37c19d9adfef1db not found: ID does not exist"
Sep 16 18:30:38.019890 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.019845    2576 scope.go:117] "RemoveContainer" containerID="45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"
Sep 16 18:30:38.020125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020100    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f"} err="failed to get container status \"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f\": rpc error: code = NotFound desc = could not find container \"45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f\": container with ID starting with 45d0b7c5d75e964a61ed749ba8382f22996c4cdb98886b1ed094b2241e381d5f not found: ID does not exist"
Sep 16 18:30:38.020188 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020127    2576 scope.go:117] "RemoveContainer" containerID="9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"
Sep 16 18:30:38.020397 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020377    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305"} err="failed to get container status \"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305\": rpc error: code = NotFound desc = could not find container \"9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305\": container with ID starting with 9fe0cb5197bd1109ca14e081c235f0495538ef800eaad0a60c8e35e1df5f4305 not found: ID does not exist"
Sep 16 18:30:38.020397 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020396    2576 scope.go:117] "RemoveContainer" containerID="46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"
Sep 16 18:30:38.020673 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020654    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a"} err="failed to get container status \"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a\": rpc error: code = NotFound desc = could not find container \"46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a\": container with ID starting with 46c7de13cc6030a6afdff2351cb74e3f3020a784da3c60338c32db39a0b3423a not found: ID does not exist"
Sep 16 18:30:38.020746 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020675    2576 scope.go:117] "RemoveContainer" containerID="ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"
Sep 16 18:30:38.020999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.020974    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7"} err="failed to get container status \"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7\": rpc error: code = NotFound desc = could not find container \"ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7\": container with ID starting with ca9332aefb86c297e963dc9c79b66c2669e118ef6f950664edadaaa7693ceff7 not found: ID does not exist"
Sep 16 18:30:38.021112 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.021001    2576 scope.go:117] "RemoveContainer" containerID="7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"
Sep 16 18:30:38.021291 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.021274    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab"} err="failed to get container status \"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab\": rpc error: code = NotFound desc = could not find container \"7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab\": container with ID starting with 7ec07f0f197678aa66f13e4d90225eee54904ce34ab74c5cf58a5c3be96e39ab not found: ID does not exist"
Sep 16 18:30:38.021366 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.021292    2576 scope.go:117] "RemoveContainer" containerID="6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"
Sep 16 18:30:38.021565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.021545    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774"} err="failed to get container status \"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774\": rpc error: code = NotFound desc = could not find container \"6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774\": container with ID starting with 6e3b34fba98685d0a23c9a0add2630f07cc2abc63f1d7d183d319d594117a774 not found: ID does not exist"
Sep 16 18:30:38.056037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056014    2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-cluster-tls-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056035    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056044    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-metrics-client-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056052    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056061    2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-volume\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056070    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056078    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056086    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-tls-assets\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056094    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-alertmanager-main-db\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056101    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822f2d94-8441-4f40-b6bd-9f9e4888cf75-config-out\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056109    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-web-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056116    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/822f2d94-8441-4f40-b6bd-9f9e4888cf75-secret-alertmanager-main-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.056143 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.056123    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwk4l\" (UniqueName: \"kubernetes.io/projected/822f2d94-8441-4f40-b6bd-9f9e4888cf75-kube-api-access-qwk4l\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:30:38.281722 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.281670    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:38.286826 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.286807    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:38.327411 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327391    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:38.327675 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327663    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-web"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327676    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-web"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327687    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="prom-label-proxy"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327693    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="prom-label-proxy"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327702    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="config-reloader"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327708    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="config-reloader"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327719    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327724    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327730    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374df99c-32ba-49d2-a155-897d231deac1" containerName="console"
Sep 16 18:30:38.327736 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327735    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="374df99c-32ba-49d2-a155-897d231deac1" containerName="console"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327742    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="alertmanager"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327747    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="alertmanager"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327756    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="init-config-reloader"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327761    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="init-config-reloader"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327768    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-metric"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327773    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-metric"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327821    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-web"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327829    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy-metric"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327835    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="kube-rbac-proxy"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327840    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="prom-label-proxy"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327845    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="alertmanager"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327850    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="374df99c-32ba-49d2-a155-897d231deac1" containerName="console"
Sep 16 18:30:38.328038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.327855    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" containerName="config-reloader"
Sep 16 18:30:38.330999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.330984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.335894 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.335867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 18:30:38.335999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.335901    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 18:30:38.335999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.335908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 18:30:38.336483 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.336463    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bkvkh\""
Sep 16 18:30:38.336658 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.336641    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 18:30:38.336745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.336643    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 18:30:38.336745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.336643    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 18:30:38.336745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.336699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 18:30:38.338980 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.338962    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 18:30:38.342705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.342689    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 18:30:38.350190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.350169    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:38.358774 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.358864 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-config-volume\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.358864 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.358864 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-config-out\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj5kr\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-kube-api-access-sj5kr\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.358994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-web-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.359014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.359037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.359057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.359073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.359133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.359125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj5kr\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-kube-api-access-sj5kr\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-web-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.459866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-config-volume\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.460162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.460162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.459914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-config-out\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.460818 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.460362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.460818 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.460542    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.460818 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.460620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e6668a-5d23-4250-8147-49be791c6824-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e6668a-5d23-4250-8147-49be791c6824-config-out\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-web-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463769 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463850 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463770    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463850 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463850 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-config-volume\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.463850 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.463811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.464898 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.464881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c8e6668a-5d23-4250-8147-49be791c6824-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.471320 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.471304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj5kr\" (UniqueName: \"kubernetes.io/projected/c8e6668a-5d23-4250-8147-49be791c6824-kube-api-access-sj5kr\") pod \"alertmanager-main-0\" (UID: \"c8e6668a-5d23-4250-8147-49be791c6824\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.640186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.640135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:30:38.931296 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.931271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:30:38.933445 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:30:38.933420    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e6668a_5d23_4250_8147_49be791c6824.slice/crio-0050270f422ca756c5c52a97fd5bd1bce4a422f71eeffd04ae37b602f3970f32 WatchSource:0}: Error finding container 0050270f422ca756c5c52a97fd5bd1bce4a422f71eeffd04ae37b602f3970f32: Status 404 returned error can't find the container with id 0050270f422ca756c5c52a97fd5bd1bce4a422f71eeffd04ae37b602f3970f32
Sep 16 18:30:38.956961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:38.956920    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"0050270f422ca756c5c52a97fd5bd1bce4a422f71eeffd04ae37b602f3970f32"}
Sep 16 18:30:39.010536 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:39.010510    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="822f2d94-8441-4f40-b6bd-9f9e4888cf75" path="/var/lib/kubelet/pods/822f2d94-8441-4f40-b6bd-9f9e4888cf75/volumes"
Sep 16 18:30:39.963196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:39.963152    2576 generic.go:356] "Generic (PLEG): container finished" podID="c8e6668a-5d23-4250-8147-49be791c6824" containerID="44f5735000a206b380feac0247f6efb1142adb86b4a208fc03e94890d41dfb0e" exitCode=0
Sep 16 18:30:39.963560 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:39.963233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerDied","Data":"44f5735000a206b380feac0247f6efb1142adb86b4a208fc03e94890d41dfb0e"}
Sep 16 18:30:40.970792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.970756    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"9d0e72ebe7371670f29139beeed48d5d4af34a4701808d7515404eed6b32b1b3"}
Sep 16 18:30:40.971208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.970835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"886bf30c762576a5e728adedf78265d7cc6698505bfd73c9952a120320631edb"}
Sep 16 18:30:40.971208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.970881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"50203341f5ca1eafaf368e313255a45e4609f0a9def200317973f33654e954b1"}
Sep 16 18:30:40.971208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.970916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"98512c09575c8a1d55285d5c6e0c5d172acc171ee23a8d812f21b5d89980888a"}
Sep 16 18:30:40.971208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.970985    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"83e40b8c2a7b6b0fd0dcb8f4e53efe57259127f85e8eb1925f1a4e7e25312235"}
Sep 16 18:30:40.971208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:40.971022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c8e6668a-5d23-4250-8147-49be791c6824","Type":"ContainerStarted","Data":"be4cd3515c80431462a16f75eb6475739731fcf39dc0c68062360520ffb7f984"}
Sep 16 18:30:41.990796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:41.990741    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.972841822 podStartE2EDuration="3.9907236s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.881778ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:41.020441894 +0000 UTC m=+168.440928765" watchObservedRunningTime="2026-09-16 18:30:41.9907236 +0000 UTC m=+169.411210459"
Sep 16 18:30:41.991616 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:41.991594    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:30:41.994088 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:41.994066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.013677 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.013653    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:30:42.087678 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.087654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.087913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.087884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.088095 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.088079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.088166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.088155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9hlx\" (UniqueName: \"kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.088268 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.088250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.088305 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.088294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.088358 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.088335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.188894 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.188871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.188902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.188918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.188965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.188985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.189001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189227 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.189028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9hlx\" (UniqueName: \"kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189619 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.189590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189746 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.189594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.189746 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.189663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.190086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.190065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.191522 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.191494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.191767 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.191744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.203262 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.203239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9hlx\" (UniqueName: \"kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx\") pod \"console-79478bcdbc-b24cc\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") " pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.302707 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.302654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:42.579593 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.579572    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:30:42.581447 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:30:42.581423    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597cdfa0_ea8a_4732_a031_f7887d14fef9.slice/crio-fa57219dd9c844d977b397f05b247c93fde95935a506bee2c6a70fc9c4eac00d WatchSource:0}: Error finding container fa57219dd9c844d977b397f05b247c93fde95935a506bee2c6a70fc9c4eac00d: Status 404 returned error can't find the container with id fa57219dd9c844d977b397f05b247c93fde95935a506bee2c6a70fc9c4eac00d
Sep 16 18:30:42.981992 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.981957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79478bcdbc-b24cc" event={"ID":"597cdfa0-ea8a-4732-a031-f7887d14fef9","Type":"ContainerStarted","Data":"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"}
Sep 16 18:30:42.982152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:42.982008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79478bcdbc-b24cc" event={"ID":"597cdfa0-ea8a-4732-a031-f7887d14fef9","Type":"ContainerStarted","Data":"fa57219dd9c844d977b397f05b247c93fde95935a506bee2c6a70fc9c4eac00d"}
Sep 16 18:30:43.006250 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:43.006200    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79478bcdbc-b24cc" podStartSLOduration=2.006181818 podStartE2EDuration="2.006181818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:43.004047306 +0000 UTC m=+170.424534166" watchObservedRunningTime="2026-09-16 18:30:43.006181818 +0000 UTC m=+170.426668680"
Sep 16 18:30:52.303350 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:52.303326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:52.303696 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:52.303402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:52.308175 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:52.308156    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:53.033069 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:53.033043    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:30:53.085119 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:30:53.085096    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:31:18.106060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.106003    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cc9f786f5-w6fpk" podUID="79ca2527-05d8-40c4-b349-6c946ba858ed" containerName="console" containerID="cri-o://171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec" gracePeriod=15
Sep 16 18:31:18.343910 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.343876    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:31:18.468791 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.468761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.468967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.468804    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.468967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.468855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.468967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.468906    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.468967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.468955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.469186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469002    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt5nc\" (UniqueName: \"kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.469186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469034    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert\") pod \"79ca2527-05d8-40c4-b349-6c946ba858ed\" (UID: \"79ca2527-05d8-40c4-b349-6c946ba858ed\") "
Sep 16 18:31:18.469314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:18.469462 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config" (OuterVolumeSpecName: "console-config") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:18.469525 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:18.469577 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.469554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:18.471237 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.471211    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:31:18.471451 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.471424    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:31:18.471565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.471466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc" (OuterVolumeSpecName: "kube-api-access-kt5nc") pod "79ca2527-05d8-40c4-b349-6c946ba858ed" (UID: "79ca2527-05d8-40c4-b349-6c946ba858ed"). InnerVolumeSpecName "kube-api-access-kt5nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:31:18.570419 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570394    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570419 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570416    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt5nc\" (UniqueName: \"kubernetes.io/projected/79ca2527-05d8-40c4-b349-6c946ba858ed-kube-api-access-kt5nc\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570425    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570434    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570442    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79ca2527-05d8-40c4-b349-6c946ba858ed-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570450    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:18.570546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:18.570458    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ca2527-05d8-40c4-b349-6c946ba858ed-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:31:19.134014 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.133981    2576 generic.go:356] "Generic (PLEG): container finished" podID="79ca2527-05d8-40c4-b349-6c946ba858ed" containerID="171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec" exitCode=0
Sep 16 18:31:19.134388 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.134027    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc9f786f5-w6fpk"
Sep 16 18:31:19.134388 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.134042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc9f786f5-w6fpk" event={"ID":"79ca2527-05d8-40c4-b349-6c946ba858ed","Type":"ContainerDied","Data":"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"}
Sep 16 18:31:19.134388 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.134091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc9f786f5-w6fpk" event={"ID":"79ca2527-05d8-40c4-b349-6c946ba858ed","Type":"ContainerDied","Data":"ec4697c4334c3b70606347159b08a3ca9b626831c530860a6cea52e310044ab2"}
Sep 16 18:31:19.134388 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.134117    2576 scope.go:117] "RemoveContainer" containerID="171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"
Sep 16 18:31:19.142699 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.142677    2576 scope.go:117] "RemoveContainer" containerID="171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"
Sep 16 18:31:19.143061 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:31:19.143032    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec\": container with ID starting with 171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec not found: ID does not exist" containerID="171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"
Sep 16 18:31:19.143154 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.143072    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec"} err="failed to get container status \"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec\": rpc error: code = NotFound desc = could not find container \"171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec\": container with ID starting with 171ac7349471b08b29269f70936455078a330aa2244808d1736c488dae6b92ec not found: ID does not exist"
Sep 16 18:31:19.153387 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.153365    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:31:19.155739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:19.155718    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cc9f786f5-w6fpk"]
Sep 16 18:31:21.009840 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:21.009804    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ca2527-05d8-40c4-b349-6c946ba858ed" path="/var/lib/kubelet/pods/79ca2527-05d8-40c4-b349-6c946ba858ed/volumes"
Sep 16 18:31:44.249093 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.249058    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"]
Sep 16 18:31:44.249468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.249416    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ca2527-05d8-40c4-b349-6c946ba858ed" containerName="console"
Sep 16 18:31:44.249468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.249427    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ca2527-05d8-40c4-b349-6c946ba858ed" containerName="console"
Sep 16 18:31:44.249536 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.249484    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ca2527-05d8-40c4-b349-6c946ba858ed" containerName="console"
Sep 16 18:31:44.251521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.251503    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.254000 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.253975    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:31:44.254807 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.254789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:31:44.254807 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.254801    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d5f6h\""
Sep 16 18:31:44.264886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.264862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"]
Sep 16 18:31:44.365053 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.365033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.365166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.365064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qnzf\" (UniqueName: \"kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.365166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.365117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.465597 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.465575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.465692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.465623    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.465692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.465645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qnzf\" (UniqueName: \"kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.465947 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.465911    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.466044 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.466020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.476573 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.476554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qnzf\" (UniqueName: \"kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.561532 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.561485    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:31:44.839701 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:44.839678    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"]
Sep 16 18:31:44.841598 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:31:44.841570    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4a5e60_8c2c_487e_8044_a3505d808537.slice/crio-0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f WatchSource:0}: Error finding container 0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f: Status 404 returned error can't find the container with id 0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f
Sep 16 18:31:45.244751 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:45.244720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerStarted","Data":"0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f"}
Sep 16 18:31:52.282579 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:52.282546    2576 generic.go:356] "Generic (PLEG): container finished" podID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerID="f098bfe855c160f52c27f76d2cd5596b17f28356206f88f21fe8f877f40b8fd0" exitCode=0
Sep 16 18:31:52.282986 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:52.282616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerDied","Data":"f098bfe855c160f52c27f76d2cd5596b17f28356206f88f21fe8f877f40b8fd0"}
Sep 16 18:31:55.296523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:55.296486    2576 generic.go:356] "Generic (PLEG): container finished" podID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerID="84e54423702e9201669cb379ef0717beb1bbe4d8e0b6fa00daaca89b26b29f27" exitCode=0
Sep 16 18:31:55.296897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:31:55.296554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerDied","Data":"84e54423702e9201669cb379ef0717beb1bbe4d8e0b6fa00daaca89b26b29f27"}
Sep 16 18:32:02.333371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:02.333333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerStarted","Data":"2a66ea404e878564a98244828a0be9c5eecaaf9aacd400001acbad1db500c1d1"}
Sep 16 18:32:02.355018 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:02.354969    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" podStartSLOduration=2.566728613 podStartE2EDuration="18.354924821s" totalImagesPullingTime="15.788196208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:02.354179203 +0000 UTC m=+249.774666087" watchObservedRunningTime="2026-09-16 18:32:02.354924821 +0000 UTC m=+249.775411682"
Sep 16 18:32:03.339806 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:03.339776    2576 generic.go:356] "Generic (PLEG): container finished" podID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerID="2a66ea404e878564a98244828a0be9c5eecaaf9aacd400001acbad1db500c1d1" exitCode=0
Sep 16 18:32:03.340190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:03.339834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerDied","Data":"2a66ea404e878564a98244828a0be9c5eecaaf9aacd400001acbad1db500c1d1"}
Sep 16 18:32:04.470918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.470894    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:32:04.534049 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.534028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle\") pod \"1d4a5e60-8c2c-487e-8044-a3505d808537\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") "
Sep 16 18:32:04.534166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.534083    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util\") pod \"1d4a5e60-8c2c-487e-8044-a3505d808537\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") "
Sep 16 18:32:04.534166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.534127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qnzf\" (UniqueName: \"kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf\") pod \"1d4a5e60-8c2c-487e-8044-a3505d808537\" (UID: \"1d4a5e60-8c2c-487e-8044-a3505d808537\") "
Sep 16 18:32:04.534864 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.534794    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle" (OuterVolumeSpecName: "bundle") pod "1d4a5e60-8c2c-487e-8044-a3505d808537" (UID: "1d4a5e60-8c2c-487e-8044-a3505d808537"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:04.536347 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.536324    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf" (OuterVolumeSpecName: "kube-api-access-5qnzf") pod "1d4a5e60-8c2c-487e-8044-a3505d808537" (UID: "1d4a5e60-8c2c-487e-8044-a3505d808537"). InnerVolumeSpecName "kube-api-access-5qnzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:04.539018 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.538994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util" (OuterVolumeSpecName: "util") pod "1d4a5e60-8c2c-487e-8044-a3505d808537" (UID: "1d4a5e60-8c2c-487e-8044-a3505d808537"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:04.635606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.635549    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:04.635606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.635572    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d4a5e60-8c2c-487e-8044-a3505d808537-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:04.635606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:04.635584    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qnzf\" (UniqueName: \"kubernetes.io/projected/1d4a5e60-8c2c-487e-8044-a3505d808537-kube-api-access-5qnzf\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:05.349196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:05.349133    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs" event={"ID":"1d4a5e60-8c2c-487e-8044-a3505d808537","Type":"ContainerDied","Data":"0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f"}
Sep 16 18:32:05.349196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:05.349177    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0494daeb3a07e01438d7b1df550ead5b4aaea5e693e81e35ca72fb6632f6632f"
Sep 16 18:32:05.349196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:05.349176    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bjmgs"
Sep 16 18:32:12.460858 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.460827    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"]
Sep 16 18:32:12.461953 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.461904    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="pull"
Sep 16 18:32:12.462269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462083    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="pull"
Sep 16 18:32:12.462269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462114    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="extract"
Sep 16 18:32:12.462269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462122    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="extract"
Sep 16 18:32:12.462269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462138    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="util"
Sep 16 18:32:12.462269 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462146    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="util"
Sep 16 18:32:12.462555 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.462343    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4a5e60-8c2c-487e-8044-a3505d808537" containerName="extract"
Sep 16 18:32:12.467044 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.467024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.469306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.469275    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-s9xvh\""
Sep 16 18:32:12.469306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.469291    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 16 18:32:12.469306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.469298    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:32:12.480020 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.480000    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"]
Sep 16 18:32:12.592005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.591980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97a329b5-a6ee-456d-8746-390da6bb5c62-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.592103 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.592010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzq7\" (UniqueName: \"kubernetes.io/projected/97a329b5-a6ee-456d-8746-390da6bb5c62-kube-api-access-8bzq7\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.692680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.692644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97a329b5-a6ee-456d-8746-390da6bb5c62-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.692680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.692686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzq7\" (UniqueName: \"kubernetes.io/projected/97a329b5-a6ee-456d-8746-390da6bb5c62-kube-api-access-8bzq7\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.693019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.693003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97a329b5-a6ee-456d-8746-390da6bb5c62-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.707641 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.707611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzq7\" (UniqueName: \"kubernetes.io/projected/97a329b5-a6ee-456d-8746-390da6bb5c62-kube-api-access-8bzq7\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4hmm\" (UID: \"97a329b5-a6ee-456d-8746-390da6bb5c62\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:12.776593 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:12.776542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"
Sep 16 18:32:13.055130 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:13.055023    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm"]
Sep 16 18:32:13.058447 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:13.058426    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a329b5_a6ee_456d_8746_390da6bb5c62.slice/crio-a2060c1106bee41e6c2f54017f6653f8ebed942ba4c85fa4ec36867ca2b32bca WatchSource:0}: Error finding container a2060c1106bee41e6c2f54017f6653f8ebed942ba4c85fa4ec36867ca2b32bca: Status 404 returned error can't find the container with id a2060c1106bee41e6c2f54017f6653f8ebed942ba4c85fa4ec36867ca2b32bca
Sep 16 18:32:13.383132 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:13.383054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm" event={"ID":"97a329b5-a6ee-456d-8746-390da6bb5c62","Type":"ContainerStarted","Data":"a2060c1106bee41e6c2f54017f6653f8ebed942ba4c85fa4ec36867ca2b32bca"}
Sep 16 18:32:15.394681 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:15.394645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm" event={"ID":"97a329b5-a6ee-456d-8746-390da6bb5c62","Type":"ContainerStarted","Data":"01bed85ce97db1dd85f5848797f7d8befcf2bc0cd42fae9c1c8f7db6968f5c1e"}
Sep 16 18:32:15.437335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:15.437291    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4hmm" podStartSLOduration=1.618125033 podStartE2EDuration="3.437275365s" totalImagesPullingTime="1.819150332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:15.436189428 +0000 UTC m=+262.856676305" watchObservedRunningTime="2026-09-16 18:32:15.437275365 +0000 UTC m=+262.857762225"
Sep 16 18:32:16.895754 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.895720    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"]
Sep 16 18:32:16.898099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.898085    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:16.900550 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.900529    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:32:16.900694 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.900677    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d5f6h\""
Sep 16 18:32:16.901379 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.901365    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:32:16.909317 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:16.909298    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"]
Sep 16 18:32:17.027915 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.027892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.028064 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.027945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.028064 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.028003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7gvb\" (UniqueName: \"kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.128821 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.128791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7gvb\" (UniqueName: \"kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.128990 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.128904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.128990 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.128977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.129319 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.129294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.129382 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.129324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.151436 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.151378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7gvb\" (UniqueName: \"kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.207306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.207288    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:17.492994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:17.492967    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"]
Sep 16 18:32:17.495609 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:17.495583    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6237a80e_281b_4a76_8dc5_11b31435106a.slice/crio-46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4 WatchSource:0}: Error finding container 46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4: Status 404 returned error can't find the container with id 46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4
Sep 16 18:32:18.409784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:18.409713    2576 generic.go:356] "Generic (PLEG): container finished" podID="6237a80e-281b-4a76-8dc5-11b31435106a" containerID="192322beb9fd7e012ee1c4eae5e8b8fce76486a272c92d753ce0a7363ec583d5" exitCode=0
Sep 16 18:32:18.409784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:18.409748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp" event={"ID":"6237a80e-281b-4a76-8dc5-11b31435106a","Type":"ContainerDied","Data":"192322beb9fd7e012ee1c4eae5e8b8fce76486a272c92d753ce0a7363ec583d5"}
Sep 16 18:32:18.409784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:18.409784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp" event={"ID":"6237a80e-281b-4a76-8dc5-11b31435106a","Type":"ContainerStarted","Data":"46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4"}
Sep 16 18:32:21.091606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.091570    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"]
Sep 16 18:32:21.093997 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.093978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.096390 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.096373    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 18:32:21.097230 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.097215    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7sgfm\""
Sep 16 18:32:21.097230 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.097222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 18:32:21.109118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.109091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"]
Sep 16 18:32:21.160357 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.160333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktg86\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-kube-api-access-ktg86\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.160444 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.160371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.261091 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.261068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktg86\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-kube-api-access-ktg86\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.261168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.261112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.272689 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.272662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktg86\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-kube-api-access-ktg86\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.273300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.273276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e2882ae-2684-400a-aa3d-34a28c67f354-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8k4dv\" (UID: \"1e2882ae-2684-400a-aa3d-34a28c67f354\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.416694 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.416675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"
Sep 16 18:32:21.426367 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.426346    2576 generic.go:356] "Generic (PLEG): container finished" podID="6237a80e-281b-4a76-8dc5-11b31435106a" containerID="26f8f02f077c449f07691407048aa3f8054762b85bc2ca65ce50848d809695eb" exitCode=0
Sep 16 18:32:21.426480 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.426460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp" event={"ID":"6237a80e-281b-4a76-8dc5-11b31435106a","Type":"ContainerDied","Data":"26f8f02f077c449f07691407048aa3f8054762b85bc2ca65ce50848d809695eb"}
Sep 16 18:32:21.702883 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:21.702863    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv"]
Sep 16 18:32:22.432651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:22.432617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv" event={"ID":"1e2882ae-2684-400a-aa3d-34a28c67f354","Type":"ContainerStarted","Data":"d8e4913db01a0bcc5d17815a56cf1e8d425bca57e8fe53ef1c344a9c432af1ff"}
Sep 16 18:32:22.434813 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:22.434786    2576 generic.go:356] "Generic (PLEG): container finished" podID="6237a80e-281b-4a76-8dc5-11b31435106a" containerID="acc8bb1ae18320383eb2810347b19c4aea2f913ab8a548e15b71400cd6985671" exitCode=0
Sep 16 18:32:22.434943 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:22.434823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp" event={"ID":"6237a80e-281b-4a76-8dc5-11b31435106a","Type":"ContainerDied","Data":"acc8bb1ae18320383eb2810347b19c4aea2f913ab8a548e15b71400cd6985671"}
Sep 16 18:32:23.640726 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.640688    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nvdkp"]
Sep 16 18:32:23.643561 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.643531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.646007 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.645987    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8xf58\""
Sep 16 18:32:23.657589 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.657569    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nvdkp"]
Sep 16 18:32:23.784802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.784712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.784802 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.784756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6wrz\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-kube-api-access-r6wrz\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.886203 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.886157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.886203 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.886197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6wrz\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-kube-api-access-r6wrz\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.896559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.896528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6wrz\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-kube-api-access-r6wrz\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.896680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.896590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207e1d7e-88c0-4161-b973-0ecbd53196a3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nvdkp\" (UID: \"207e1d7e-88c0-4161-b973-0ecbd53196a3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:23.955119 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:23.955090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:24.440130 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.440107    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:24.448598 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.448579    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp"
Sep 16 18:32:24.448598 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.448582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7c9wp" event={"ID":"6237a80e-281b-4a76-8dc5-11b31435106a","Type":"ContainerDied","Data":"46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4"}
Sep 16 18:32:24.448807 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.448647    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46bfbec30bb46489433075866227e92c902ef1f68186439617ea9ab17cec86e4"
Sep 16 18:32:24.593035 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.593013    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util\") pod \"6237a80e-281b-4a76-8dc5-11b31435106a\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") "
Sep 16 18:32:24.593169 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.593094    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle\") pod \"6237a80e-281b-4a76-8dc5-11b31435106a\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") "
Sep 16 18:32:24.593169 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.593120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7gvb\" (UniqueName: \"kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb\") pod \"6237a80e-281b-4a76-8dc5-11b31435106a\" (UID: \"6237a80e-281b-4a76-8dc5-11b31435106a\") "
Sep 16 18:32:24.593426 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.593404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle" (OuterVolumeSpecName: "bundle") pod "6237a80e-281b-4a76-8dc5-11b31435106a" (UID: "6237a80e-281b-4a76-8dc5-11b31435106a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:24.595247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.595223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb" (OuterVolumeSpecName: "kube-api-access-b7gvb") pod "6237a80e-281b-4a76-8dc5-11b31435106a" (UID: "6237a80e-281b-4a76-8dc5-11b31435106a"). InnerVolumeSpecName "kube-api-access-b7gvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:24.598793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.598771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util" (OuterVolumeSpecName: "util") pod "6237a80e-281b-4a76-8dc5-11b31435106a" (UID: "6237a80e-281b-4a76-8dc5-11b31435106a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:24.694381 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.694355    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:24.694381 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.694378    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6237a80e-281b-4a76-8dc5-11b31435106a-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:24.694738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.694388    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7gvb\" (UniqueName: \"kubernetes.io/projected/6237a80e-281b-4a76-8dc5-11b31435106a-kube-api-access-b7gvb\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:24.707853 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:24.707823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod207e1d7e_88c0_4161_b973_0ecbd53196a3.slice/crio-40d4415f1d030c051e808767609298af097ecda58dd917ee2ccf332d5243d513 WatchSource:0}: Error finding container 40d4415f1d030c051e808767609298af097ecda58dd917ee2ccf332d5243d513: Status 404 returned error can't find the container with id 40d4415f1d030c051e808767609298af097ecda58dd917ee2ccf332d5243d513
Sep 16 18:32:24.710569 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:24.710547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nvdkp"]
Sep 16 18:32:25.456787 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.456749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp" event={"ID":"207e1d7e-88c0-4161-b973-0ecbd53196a3","Type":"ContainerStarted","Data":"c6288058352c85d9a00db94df51370b29dd4a3bdb8c65f43166b50af74b7e6ae"}
Sep 16 18:32:25.457005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.456799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp" event={"ID":"207e1d7e-88c0-4161-b973-0ecbd53196a3","Type":"ContainerStarted","Data":"40d4415f1d030c051e808767609298af097ecda58dd917ee2ccf332d5243d513"}
Sep 16 18:32:25.457005 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.456821    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:25.458968 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.458912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv" event={"ID":"1e2882ae-2684-400a-aa3d-34a28c67f354","Type":"ContainerStarted","Data":"2b5afd94e93ce94132d2f599dcc7d3bf37d7ce44b0e3e8c22121992c2974e552"}
Sep 16 18:32:25.477763 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.477722    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp" podStartSLOduration=2.477708019 podStartE2EDuration="2.477708019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:25.476916388 +0000 UTC m=+272.897403258" watchObservedRunningTime="2026-09-16 18:32:25.477708019 +0000 UTC m=+272.898194878"
Sep 16 18:32:25.500900 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:25.500854    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8k4dv" podStartSLOduration=1.750983008 podStartE2EDuration="4.500840787s" totalImagesPullingTime="2.749857779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:25.499191204 +0000 UTC m=+272.919678066" watchObservedRunningTime="2026-09-16 18:32:25.500840787 +0000 UTC m=+272.921327646"
Sep 16 18:32:30.105185 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105155    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"]
Sep 16 18:32:30.105516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105492    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="extract"
Sep 16 18:32:30.105516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105503    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="extract"
Sep 16 18:32:30.105516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105518    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="pull"
Sep 16 18:32:30.105624 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105524    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="pull"
Sep 16 18:32:30.105624 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105540    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="util"
Sep 16 18:32:30.105624 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105545    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="util"
Sep 16 18:32:30.105624 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.105593    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6237a80e-281b-4a76-8dc5-11b31435106a" containerName="extract"
Sep 16 18:32:30.156295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.156270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"]
Sep 16 18:32:30.156434 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.156350    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.159821 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.159798    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:32:30.159952 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.159867    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 18:32:30.160763 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.160749    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mvjtr\""
Sep 16 18:32:30.235972 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.235947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8zw9\" (UniqueName: \"kubernetes.io/projected/e1a0afb1-e344-41b7-b971-7dc48c5f3820-kube-api-access-w8zw9\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.236083 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.235986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0afb1-e344-41b7-b971-7dc48c5f3820-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.336849 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.336824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8zw9\" (UniqueName: \"kubernetes.io/projected/e1a0afb1-e344-41b7-b971-7dc48c5f3820-kube-api-access-w8zw9\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.336961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.336864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0afb1-e344-41b7-b971-7dc48c5f3820-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.337172 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.337158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0afb1-e344-41b7-b971-7dc48c5f3820-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.348258 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.348237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8zw9\" (UniqueName: \"kubernetes.io/projected/e1a0afb1-e344-41b7-b971-7dc48c5f3820-kube-api-access-w8zw9\") pod \"openshift-lws-operator-fd8ccff4c-ltxmj\" (UID: \"e1a0afb1-e344-41b7-b971-7dc48c5f3820\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.465871 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.465847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"
Sep 16 18:32:30.783954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:30.783911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj"]
Sep 16 18:32:30.786605 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:30.786576    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1a0afb1_e344_41b7_b971_7dc48c5f3820.slice/crio-9445cda3631007fd74129fb8c93afbe2bbb96d415117fa84b2e0a7bc592d3c46 WatchSource:0}: Error finding container 9445cda3631007fd74129fb8c93afbe2bbb96d415117fa84b2e0a7bc592d3c46: Status 404 returned error can't find the container with id 9445cda3631007fd74129fb8c93afbe2bbb96d415117fa84b2e0a7bc592d3c46
Sep 16 18:32:31.465346 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:31.465314    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nvdkp"
Sep 16 18:32:31.488805 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:31.488775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj" event={"ID":"e1a0afb1-e344-41b7-b971-7dc48c5f3820","Type":"ContainerStarted","Data":"9445cda3631007fd74129fb8c93afbe2bbb96d415117fa84b2e0a7bc592d3c46"}
Sep 16 18:32:33.501342 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:33.501274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj" event={"ID":"e1a0afb1-e344-41b7-b971-7dc48c5f3820","Type":"ContainerStarted","Data":"aaaa0ce4f4a54fab1dffa0a7f21293b34f3bcf2dea7c4387d2a63654a408fca5"}
Sep 16 18:32:33.536162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:33.536113    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ltxmj" podStartSLOduration=1.126828001 podStartE2EDuration="3.53609532s" totalImagesPullingTime="2.409267319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:33.534120784 +0000 UTC m=+280.954607644" watchObservedRunningTime="2026-09-16 18:32:33.53609532 +0000 UTC m=+280.956582180"
Sep 16 18:32:36.030889 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.030854    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"]
Sep 16 18:32:36.033623 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.033608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.036170 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.036147    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d5f6h\""
Sep 16 18:32:36.036297 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.036222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:32:36.037041 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.037027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:32:36.052674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.052651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"]
Sep 16 18:32:36.183602 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.183574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.183737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.183604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.183737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.183628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lv2c\" (UniqueName: \"kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.284840 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.284779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lv2c\" (UniqueName: \"kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.284961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.284852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.284961 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.284871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.285195 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.285178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.285273 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.285234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.296463 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.296434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lv2c\" (UniqueName: \"kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.343127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.343105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:36.625249 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:36.625223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"]
Sep 16 18:32:36.626841 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:36.626806    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0c0e84_e39f_4e31_9827_1df1c38c4ccc.slice/crio-70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41 WatchSource:0}: Error finding container 70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41: Status 404 returned error can't find the container with id 70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41
Sep 16 18:32:37.520432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:37.520398    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerID="5c1e94b4a46629cda8985012048d864148d7333f39345f4f57bf43ecc03327aa" exitCode=0
Sep 16 18:32:37.520764 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:37.520486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx" event={"ID":"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc","Type":"ContainerDied","Data":"5c1e94b4a46629cda8985012048d864148d7333f39345f4f57bf43ecc03327aa"}
Sep 16 18:32:37.520764 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:37.520539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx" event={"ID":"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc","Type":"ContainerStarted","Data":"70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41"}
Sep 16 18:32:38.527732 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:38.527699    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerID="562defe1a36f11448b9eb649fde66fb157c9351e875f9132d523bf740956d844" exitCode=0
Sep 16 18:32:38.528082 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:38.527756    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx" event={"ID":"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc","Type":"ContainerDied","Data":"562defe1a36f11448b9eb649fde66fb157c9351e875f9132d523bf740956d844"}
Sep 16 18:32:39.534720 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:39.534683    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerID="c0675862c7e98ef6fb00e71f12b8ccfdf99e8981210a1f7148953aa550ba5bc4" exitCode=0
Sep 16 18:32:39.535162 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:39.534761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx" event={"ID":"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc","Type":"ContainerDied","Data":"c0675862c7e98ef6fb00e71f12b8ccfdf99e8981210a1f7148953aa550ba5bc4"}
Sep 16 18:32:40.668745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.668723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:40.823999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.823897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle\") pod \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") "
Sep 16 18:32:40.823999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.823973    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util\") pod \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") "
Sep 16 18:32:40.824224 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.824053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lv2c\" (UniqueName: \"kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c\") pod \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\" (UID: \"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc\") "
Sep 16 18:32:40.824739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.824697    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle" (OuterVolumeSpecName: "bundle") pod "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" (UID: "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:40.826218 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.826193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c" (OuterVolumeSpecName: "kube-api-access-8lv2c") pod "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" (UID: "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc"). InnerVolumeSpecName "kube-api-access-8lv2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:40.829521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.829475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util" (OuterVolumeSpecName: "util") pod "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" (UID: "ee0c0e84-e39f-4e31-9827-1df1c38c4ccc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:40.925516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.925480    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lv2c\" (UniqueName: \"kubernetes.io/projected/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-kube-api-access-8lv2c\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:40.925516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.925511    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:40.925516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:40.925523    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee0c0e84-e39f-4e31-9827-1df1c38c4ccc-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:41.545355 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:41.545325    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx"
Sep 16 18:32:41.545540 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:41.545366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj4xfx" event={"ID":"ee0c0e84-e39f-4e31-9827-1df1c38c4ccc","Type":"ContainerDied","Data":"70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41"}
Sep 16 18:32:41.545540 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:41.545418    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f0cbfe38a094db91b168fe300cffd1a3f93c6a317965ddfe942a6f5a355e41"
Sep 16 18:32:47.194591 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194563    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"]
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194902    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="util"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194914    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="util"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194925    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="pull"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194953    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="pull"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194962    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="extract"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.194967    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="extract"
Sep 16 18:32:47.195040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.195030    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0c0e84-e39f-4e31-9827-1df1c38c4ccc" containerName="extract"
Sep 16 18:32:47.197132 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.197113    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.200551 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.200523    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:32:47.200654 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.200611    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:32:47.201428 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.201413    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d5f6h\""
Sep 16 18:32:47.225718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.225697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"]
Sep 16 18:32:47.372847 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.372818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh9jd\" (UniqueName: \"kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.373003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.372884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.373003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.372925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.474087 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.474026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.474087 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.474058    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.474212 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.474087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh9jd\" (UniqueName: \"kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.474479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.474457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.474518 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.474472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.491520 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.491494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh9jd\" (UniqueName: \"kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.506320 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.506301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:47.806464 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:47.806440    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"]
Sep 16 18:32:47.808664 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:47.808636    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a60c51_1fd8_4a53_9b21_98c7feae3f40.slice/crio-6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640 WatchSource:0}: Error finding container 6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640: Status 404 returned error can't find the container with id 6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640
Sep 16 18:32:48.580997 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:48.580963    2576 generic.go:356] "Generic (PLEG): container finished" podID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerID="1287a909c3f3f0b005bb0b4c16fc49afcf3bb22291477f4a3afd5eebae5a8be8" exitCode=0
Sep 16 18:32:48.581384 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:48.581055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w" event={"ID":"92a60c51-1fd8-4a53-9b21-98c7feae3f40","Type":"ContainerDied","Data":"1287a909c3f3f0b005bb0b4c16fc49afcf3bb22291477f4a3afd5eebae5a8be8"}
Sep 16 18:32:48.581384 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:48.581110    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w" event={"ID":"92a60c51-1fd8-4a53-9b21-98c7feae3f40","Type":"ContainerStarted","Data":"6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640"}
Sep 16 18:32:49.422492 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.422465    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"]
Sep 16 18:32:49.424644 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.424628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.443243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.443224    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-srjkf\""
Sep 16 18:32:49.443498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.443483    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 18:32:49.466114 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.466093    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 18:32:49.468098 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.468074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 18:32:49.468167 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.468105    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 18:32:49.479369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.479345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"]
Sep 16 18:32:49.587489 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.587454    2576 generic.go:356] "Generic (PLEG): container finished" podID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerID="c8e57205155b22551df0bf695ddd1bd8e53c156124ddb27a83d44fb6e29027dd" exitCode=0
Sep 16 18:32:49.587866 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.587543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w" event={"ID":"92a60c51-1fd8-4a53-9b21-98c7feae3f40","Type":"ContainerDied","Data":"c8e57205155b22551df0bf695ddd1bd8e53c156124ddb27a83d44fb6e29027dd"}
Sep 16 18:32:49.589610 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.589584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.589714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.589620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.589757 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.589715    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.589792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.589757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2z9\" (UniqueName: \"kubernetes.io/projected/59855f7f-7943-459d-affd-eb22be5ebc6c-kube-api-access-rh2z9\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.589826 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.589787    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.690750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.690656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.690750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.690704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.691003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.690800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.691003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.690839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2z9\" (UniqueName: \"kubernetes.io/projected/59855f7f-7943-459d-affd-eb22be5ebc6c-kube-api-access-rh2z9\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.691003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.690870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.691295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.691269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.693206 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.693183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59855f7f-7943-459d-affd-eb22be5ebc6c-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.693568 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.693549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.693676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.693655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59855f7f-7943-459d-affd-eb22be5ebc6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.702348 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.702327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2z9\" (UniqueName: \"kubernetes.io/projected/59855f7f-7943-459d-affd-eb22be5ebc6c-kube-api-access-rh2z9\") pod \"opendatahub-operator-controller-manager-744584488c-zjsgc\" (UID: \"59855f7f-7943-459d-affd-eb22be5ebc6c\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:49.734730 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:49.734711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:50.021259 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:50.021223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"]
Sep 16 18:32:50.022225 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:32:50.022206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59855f7f_7943_459d_affd_eb22be5ebc6c.slice/crio-eb03e2cb7c3dfe84f3f5832d4cef8f9f63954b42382f7ce25139d3f90130e66a WatchSource:0}: Error finding container eb03e2cb7c3dfe84f3f5832d4cef8f9f63954b42382f7ce25139d3f90130e66a: Status 404 returned error can't find the container with id eb03e2cb7c3dfe84f3f5832d4cef8f9f63954b42382f7ce25139d3f90130e66a
Sep 16 18:32:50.595245 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:50.595200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc" event={"ID":"59855f7f-7943-459d-affd-eb22be5ebc6c","Type":"ContainerStarted","Data":"eb03e2cb7c3dfe84f3f5832d4cef8f9f63954b42382f7ce25139d3f90130e66a"}
Sep 16 18:32:50.598578 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:50.598398    2576 generic.go:356] "Generic (PLEG): container finished" podID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerID="5f8e30592c38b348f63d0a600826d01b07cf85a7a729ab08d6a4984361b6651c" exitCode=0
Sep 16 18:32:50.598578 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:50.598498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w" event={"ID":"92a60c51-1fd8-4a53-9b21-98c7feae3f40","Type":"ContainerDied","Data":"5f8e30592c38b348f63d0a600826d01b07cf85a7a729ab08d6a4984361b6651c"}
Sep 16 18:32:53.050828 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.023334    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:53.050828 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.037176    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:32:53.115167 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.115142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh9jd\" (UniqueName: \"kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd\") pod \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") "
Sep 16 18:32:53.115280 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.115185    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle\") pod \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") "
Sep 16 18:32:53.115280 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.115243    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util\") pod \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\" (UID: \"92a60c51-1fd8-4a53-9b21-98c7feae3f40\") "
Sep 16 18:32:53.116003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.115978    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle" (OuterVolumeSpecName: "bundle") pod "92a60c51-1fd8-4a53-9b21-98c7feae3f40" (UID: "92a60c51-1fd8-4a53-9b21-98c7feae3f40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:53.117724 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.117701    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd" (OuterVolumeSpecName: "kube-api-access-xh9jd") pod "92a60c51-1fd8-4a53-9b21-98c7feae3f40" (UID: "92a60c51-1fd8-4a53-9b21-98c7feae3f40"). InnerVolumeSpecName "kube-api-access-xh9jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:53.120224 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.120204    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util" (OuterVolumeSpecName: "util") pod "92a60c51-1fd8-4a53-9b21-98c7feae3f40" (UID: "92a60c51-1fd8-4a53-9b21-98c7feae3f40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:53.216247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.216223    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:53.216247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.216244    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh9jd\" (UniqueName: \"kubernetes.io/projected/92a60c51-1fd8-4a53-9b21-98c7feae3f40-kube-api-access-xh9jd\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:53.216371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.216255    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92a60c51-1fd8-4a53-9b21-98c7feae3f40-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:32:53.618117 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.618081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc" event={"ID":"59855f7f-7943-459d-affd-eb22be5ebc6c","Type":"ContainerStarted","Data":"2b1a29f0c1a09c755e10293456a5d0c7bfd66f81b731c4d14ad69a80e3ab10dc"}
Sep 16 18:32:53.620152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.620120    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w" event={"ID":"92a60c51-1fd8-4a53-9b21-98c7feae3f40","Type":"ContainerDied","Data":"6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640"}
Sep 16 18:32:53.620303 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.620174    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a65037dd67114e793ea7a75b214176a0c79c18fb713feabd0e96992cb2e6640"
Sep 16 18:32:53.620303 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:53.620135    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m2q5w"
Sep 16 18:32:54.626039 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:54.625970    2576 generic.go:356] "Generic (PLEG): container finished" podID="59855f7f-7943-459d-affd-eb22be5ebc6c" containerID="2b1a29f0c1a09c755e10293456a5d0c7bfd66f81b731c4d14ad69a80e3ab10dc" exitCode=0
Sep 16 18:32:54.626374 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:54.626051    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc" event={"ID":"59855f7f-7943-459d-affd-eb22be5ebc6c","Type":"ContainerDied","Data":"2b1a29f0c1a09c755e10293456a5d0c7bfd66f81b731c4d14ad69a80e3ab10dc"}
Sep 16 18:32:54.626497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:54.626483    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:32:55.633320 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:55.633239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc" event={"ID":"59855f7f-7943-459d-affd-eb22be5ebc6c","Type":"ContainerStarted","Data":"f105a9a573ed9776499d150fa0ca2b7f3121095f67bb3ffc11190ddf6a8898e0"}
Sep 16 18:32:55.633663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:55.633356    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:32:55.668429 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:32:55.668382    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc" podStartSLOduration=1.950473996 podStartE2EDuration="6.66836596s" totalImagesPullingTime="3.62667249s" totalInitContainerRuntime="1.091219474s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:55.666142741 +0000 UTC m=+303.086629602" watchObservedRunningTime="2026-09-16 18:32:55.66836596 +0000 UTC m=+303.088852818"
Sep 16 18:33:06.644437 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:06.644407    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-zjsgc"
Sep 16 18:33:09.511522 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511482    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"]
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511830    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="extract"
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511841    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="extract"
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511855    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="util"
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511860    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="util"
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511871    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="pull"
Sep 16 18:33:09.511875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511876    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="pull"
Sep 16 18:33:09.512114 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.511940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a60c51-1fd8-4a53-9b21-98c7feae3f40" containerName="extract"
Sep 16 18:33:09.516796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.516777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.521420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.521403    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 18:33:09.522393 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.522376    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 18:33:09.522393 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.522383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 18:33:09.523585 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.523565    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mm8sp\""
Sep 16 18:33:09.543643 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.543615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"]
Sep 16 18:33:09.643898 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.643865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.644061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.643913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e453022e-db65-416a-9965-cfd2fd8c3545-manager-config\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.644061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.643996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrkzl\" (UniqueName: \"kubernetes.io/projected/e453022e-db65-416a-9965-cfd2fd8c3545-kube-api-access-rrkzl\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.644061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.644037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-metrics-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.745094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.745062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e453022e-db65-416a-9965-cfd2fd8c3545-manager-config\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.745260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.745104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrkzl\" (UniqueName: \"kubernetes.io/projected/e453022e-db65-416a-9965-cfd2fd8c3545-kube-api-access-rrkzl\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.745260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.745125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-metrics-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.745260 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.745159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.745809 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.745781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e453022e-db65-416a-9965-cfd2fd8c3545-manager-config\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.747716 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.747694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.747844 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.747823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e453022e-db65-416a-9965-cfd2fd8c3545-metrics-cert\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.762309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.762261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrkzl\" (UniqueName: \"kubernetes.io/projected/e453022e-db65-416a-9965-cfd2fd8c3545-kube-api-access-rrkzl\") pod \"lws-controller-manager-75b97559b-n56jm\" (UID: \"e453022e-db65-416a-9965-cfd2fd8c3545\") " pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:09.826075 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:09.826043    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:10.160520 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:10.160434    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"]
Sep 16 18:33:10.165898 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:33:10.165865    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode453022e_db65_416a_9965_cfd2fd8c3545.slice/crio-bcf5b4ac65b4ef1cc0626abdfa005a9c6c64fbd336423c294d6aeebe1c2196dd WatchSource:0}: Error finding container bcf5b4ac65b4ef1cc0626abdfa005a9c6c64fbd336423c294d6aeebe1c2196dd: Status 404 returned error can't find the container with id bcf5b4ac65b4ef1cc0626abdfa005a9c6c64fbd336423c294d6aeebe1c2196dd
Sep 16 18:33:10.714795 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:10.714764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm" event={"ID":"e453022e-db65-416a-9965-cfd2fd8c3545","Type":"ContainerStarted","Data":"bcf5b4ac65b4ef1cc0626abdfa005a9c6c64fbd336423c294d6aeebe1c2196dd"}
Sep 16 18:33:11.721079 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:11.721045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm" event={"ID":"e453022e-db65-416a-9965-cfd2fd8c3545","Type":"ContainerStarted","Data":"915fa17455eeef1e61735b6348b51be4db1c09a51ccb612a8e64dcb5caabf9fe"}
Sep 16 18:33:11.721439 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:11.721110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:11.777504 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:11.777455    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm" podStartSLOduration=1.54225439 podStartE2EDuration="2.777439063s" totalImagesPullingTime="1.235184673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:11.775436174 +0000 UTC m=+319.195923033" watchObservedRunningTime="2026-09-16 18:33:11.777439063 +0000 UTC m=+319.197925922"
Sep 16 18:33:20.042308 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.042273    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv5wh"]
Sep 16 18:33:20.044729 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.044714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.047942 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.047908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 16 18:33:20.048056 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.047979    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kmnl4\""
Sep 16 18:33:20.069674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.069649    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv5wh"]
Sep 16 18:33:20.127748 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.127721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.127862 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.127766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8vs\" (UniqueName: \"kubernetes.io/projected/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-kube-api-access-mh8vs\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.228948 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.228904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.229069 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.228952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8vs\" (UniqueName: \"kubernetes.io/projected/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-kube-api-access-mh8vs\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.229069 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:33:20.229044    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 16 18:33:20.229149 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:33:20.229129    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert podName:51fa60a8-66ab-47f1-8b7e-ac798e054f4f nodeName:}" failed. No retries permitted until 2026-09-16 18:33:20.729109795 +0000 UTC m=+328.149596645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert") pod "odh-model-controller-7998c67dc4-mv5wh" (UID: "51fa60a8-66ab-47f1-8b7e-ac798e054f4f") : secret "odh-model-controller-webhook-cert" not found
Sep 16 18:33:20.251287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.251267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8vs\" (UniqueName: \"kubernetes.io/projected/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-kube-api-access-mh8vs\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.733431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.733407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.735999 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.735980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fa60a8-66ab-47f1-8b7e-ac798e054f4f-cert\") pod \"odh-model-controller-7998c67dc4-mv5wh\" (UID: \"51fa60a8-66ab-47f1-8b7e-ac798e054f4f\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:20.954506 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:20.954477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:21.299797 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.299680    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv5wh"]
Sep 16 18:33:21.302558 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:33:21.302534    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fa60a8_66ab_47f1_8b7e_ac798e054f4f.slice/crio-3c63b34e9a19368d05713ccadb7caf2a9239eb684ce664344dad53a21a4b9f2b WatchSource:0}: Error finding container 3c63b34e9a19368d05713ccadb7caf2a9239eb684ce664344dad53a21a4b9f2b: Status 404 returned error can't find the container with id 3c63b34e9a19368d05713ccadb7caf2a9239eb684ce664344dad53a21a4b9f2b
Sep 16 18:33:21.474379 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.474338    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9bvb7"]
Sep 16 18:33:21.477392 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.477377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.483051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.483029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 16 18:33:21.504718 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.504698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9bvb7"]
Sep 16 18:33:21.540663 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.540636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6acde495-51ab-4b52-8a17-3ecd6e634897-tls-certs\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.540764 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.540678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/6acde495-51ab-4b52-8a17-3ecd6e634897-kube-api-access-d6h26\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.641469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.641408    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6acde495-51ab-4b52-8a17-3ecd6e634897-tls-certs\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.641469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.641441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/6acde495-51ab-4b52-8a17-3ecd6e634897-kube-api-access-d6h26\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.643876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.643859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6acde495-51ab-4b52-8a17-3ecd6e634897-tls-certs\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.655080 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.655054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/6acde495-51ab-4b52-8a17-3ecd6e634897-kube-api-access-d6h26\") pod \"model-serving-api-54d89bd5fc-9bvb7\" (UID: \"6acde495-51ab-4b52-8a17-3ecd6e634897\") " pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:21.773232 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.773207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh" event={"ID":"51fa60a8-66ab-47f1-8b7e-ac798e054f4f","Type":"ContainerStarted","Data":"3c63b34e9a19368d05713ccadb7caf2a9239eb684ce664344dad53a21a4b9f2b"}
Sep 16 18:33:21.787344 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:21.787322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:22.094701 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:22.094673    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9bvb7"]
Sep 16 18:33:22.096244 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:33:22.096213    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acde495_51ab_4b52_8a17_3ecd6e634897.slice/crio-a13aedf8b86cff8eb756ecb06142c5f2c21572a6279b21aa20831d9a563beca0 WatchSource:0}: Error finding container a13aedf8b86cff8eb756ecb06142c5f2c21572a6279b21aa20831d9a563beca0: Status 404 returned error can't find the container with id a13aedf8b86cff8eb756ecb06142c5f2c21572a6279b21aa20831d9a563beca0
Sep 16 18:33:22.729405 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:22.729373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75b97559b-n56jm"
Sep 16 18:33:22.781496 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:22.781452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7" event={"ID":"6acde495-51ab-4b52-8a17-3ecd6e634897","Type":"ContainerStarted","Data":"a13aedf8b86cff8eb756ecb06142c5f2c21572a6279b21aa20831d9a563beca0"}
Sep 16 18:33:25.799824 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:25.799783    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7" event={"ID":"6acde495-51ab-4b52-8a17-3ecd6e634897","Type":"ContainerStarted","Data":"3ff10d1867c8a66492dc86160b04c20c97d99caad5997d1bf53b5f4bcfa3a6e7"}
Sep 16 18:33:25.800286 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:25.799849    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:33:25.801557 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:25.801533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh" event={"ID":"51fa60a8-66ab-47f1-8b7e-ac798e054f4f","Type":"ContainerStarted","Data":"915861d205a3d87663b25ce4c1564066b35e9fb7d6e0264cb37cc77ae69bcfee"}
Sep 16 18:33:25.801672 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:25.801579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:25.870570 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:25.870518    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7" podStartSLOduration=1.7480272000000001 podStartE2EDuration="4.870503469s" totalImagesPullingTime="3.122476269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:25.869397728 +0000 UTC m=+333.289884598" watchObservedRunningTime="2026-09-16 18:33:25.870503469 +0000 UTC m=+333.290990569"
Sep 16 18:33:28.157113 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.157058    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh" podStartSLOduration=5.253026037 podStartE2EDuration="9.157041119s" totalImagesPullingTime="3.904015082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:25.920873154 +0000 UTC m=+333.341360025" watchObservedRunningTime="2026-09-16 18:33:28.157041119 +0000 UTC m=+335.577527979"
Sep 16 18:33:28.158434 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.158407    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"]
Sep 16 18:33:28.163317 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.163298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.173803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.173773    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 18:33:28.175295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.175274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 18:33:28.175390 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.175358    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 18:33:28.175434 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.175388    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:33:28.176148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.176132    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vgn5b\""
Sep 16 18:33:28.183345 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.183326    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 18:33:28.183410 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.183352    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:33:28.239835 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.239815    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"]
Sep 16 18:33:28.300385 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300525 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee0d139-65db-4125-9880-d74fb265c56e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300525 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300525 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300525 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300686 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkl4b\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-kube-api-access-wkl4b\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.300686 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.300591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401510 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee0d139-65db-4125-9880-d74fb265c56e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401645 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.401854 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.401795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkl4b\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-kube-api-access-wkl4b\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.402521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.402484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.404401 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.404380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.404581 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.404561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.404652 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.404564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee0d139-65db-4125-9880-d74fb265c56e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.404652 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.404631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ee0d139-65db-4125-9880-d74fb265c56e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.436893 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.436866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkl4b\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-kube-api-access-wkl4b\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.444304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.444282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee0d139-65db-4125-9880-d74fb265c56e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h9gc2\" (UID: \"7ee0d139-65db-4125-9880-d74fb265c56e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.472672 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.472649    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:28.805347 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.805314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"]
Sep 16 18:33:28.806957 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:33:28.806915    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee0d139_65db_4125_9880_d74fb265c56e.slice/crio-579c58e926a0ee656370417a68d5a10468f3be13d7a12cf39f30caaffdecf6c0 WatchSource:0}: Error finding container 579c58e926a0ee656370417a68d5a10468f3be13d7a12cf39f30caaffdecf6c0: Status 404 returned error can't find the container with id 579c58e926a0ee656370417a68d5a10468f3be13d7a12cf39f30caaffdecf6c0
Sep 16 18:33:28.818523 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:28.818494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2" event={"ID":"7ee0d139-65db-4125-9880-d74fb265c56e","Type":"ContainerStarted","Data":"579c58e926a0ee656370417a68d5a10468f3be13d7a12cf39f30caaffdecf6c0"}
Sep 16 18:33:31.187407 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:31.187369    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:33:31.187647 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:31.187443    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:33:31.838227 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:31.838198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2" event={"ID":"7ee0d139-65db-4125-9880-d74fb265c56e","Type":"ContainerStarted","Data":"15687ddf18c2bbffa1c284ee537fd7d3b0bda2e14786734ad349e953537814f8"}
Sep 16 18:33:31.838407 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:31.838328    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:31.912884 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:31.912829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2" podStartSLOduration=1.5352301270000002 podStartE2EDuration="3.91281266s" totalImagesPullingTime="2.377582533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:31.911501501 +0000 UTC m=+339.331988362" watchObservedRunningTime="2026-09-16 18:33:31.91281266 +0000 UTC m=+339.333299520"
Sep 16 18:33:32.846030 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:32.845962    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h9gc2"
Sep 16 18:33:36.809691 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:36.809657    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mv5wh"
Sep 16 18:33:36.811537 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:33:36.811517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9bvb7"
Sep 16 18:34:27.112837 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.112806    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"]
Sep 16 18:34:27.115611 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.115583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.121867 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.121834    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d5f6h\""
Sep 16 18:34:27.121867 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.121835    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:34:27.122592 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.122578    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:34:27.152132 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.152102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.152251 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.152160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.152251 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.152222    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjh92\" (UniqueName: \"kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.210271 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.210243    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"]
Sep 16 18:34:27.252582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.252561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.252689 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.252607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.252689 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.252637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjh92\" (UniqueName: \"kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.252924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.252908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.252994 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.252956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.280540 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.280515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjh92\" (UniqueName: \"kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.303860 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.303838    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"]
Sep 16 18:34:27.306521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.306508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.329666 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.329645    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"]
Sep 16 18:34:27.353088 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.353047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.353219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.353097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqwbr\" (UniqueName: \"kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.353219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.353142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.425126 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.425105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:27.453044 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.453019    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"]
Sep 16 18:34:27.453817 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.453794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwbr\" (UniqueName: \"kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.453921 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.453851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.454016 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.453999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.454213 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.454190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.454335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.454318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.457302 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.457284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.476414 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.476254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwbr\" (UniqueName: \"kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.481868 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.481790    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"]
Sep 16 18:34:27.555174 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.555142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgxjm\" (UniqueName: \"kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.555295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.555196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.555295 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.555270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.615977 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.615952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:27.656223 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.656199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.656364 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.656262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxjm\" (UniqueName: \"kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.656364 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.656313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.656595 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.656577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.656654 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.656603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.675432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.675379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxjm\" (UniqueName: \"kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.687045 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.687019    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"]
Sep 16 18:34:27.690795 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.690778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.719402 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.719380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"]
Sep 16 18:34:27.747535 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.747515    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"]
Sep 16 18:34:27.749241 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:34:27.749215    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dccfa63_2b22_4d4f_9793_12f5a021a3ed.slice/crio-a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943 WatchSource:0}: Error finding container a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943: Status 404 returned error can't find the container with id a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943
Sep 16 18:34:27.757036 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.757019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.757108 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.757084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjg54\" (UniqueName: \"kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.757191 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.757170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.766816 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.766796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:27.858624 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.858571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjg54\" (UniqueName: \"kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.858784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.858661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.858784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.858701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.859222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.859196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.859222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.859211    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.871507 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.871485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjg54\" (UniqueName: \"kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:27.912201 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:27.912180    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"]
Sep 16 18:34:27.920821 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:34:27.920800    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd44afc0_2f78_4e17_8530_5b2d73d4e960.slice/crio-b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94 WatchSource:0}: Error finding container b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94: Status 404 returned error can't find the container with id b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94
Sep 16 18:34:28.000280 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.000249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:28.087511 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.087482    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"]
Sep 16 18:34:28.088474 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:34:28.088451    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e68d3c_4c06_498e_8efe_d38b9a323c3b.slice/crio-c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff WatchSource:0}: Error finding container c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff: Status 404 returned error can't find the container with id c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff
Sep 16 18:34:28.134217 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.134193    2576 generic.go:356] "Generic (PLEG): container finished" podID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerID="42445d6856e5be7914445ea2b47eb9a0aaae572be35b90622b48037ca412ca9c" exitCode=0
Sep 16 18:34:28.134533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.134234    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4" event={"ID":"cd44afc0-2f78-4e17-8530-5b2d73d4e960","Type":"ContainerDied","Data":"42445d6856e5be7914445ea2b47eb9a0aaae572be35b90622b48037ca412ca9c"}
Sep 16 18:34:28.134533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.134289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4" event={"ID":"cd44afc0-2f78-4e17-8530-5b2d73d4e960","Type":"ContainerStarted","Data":"b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94"}
Sep 16 18:34:28.136172 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.136152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerStarted","Data":"c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff"}
Sep 16 18:34:28.137986 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.137829    2576 generic.go:356] "Generic (PLEG): container finished" podID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerID="dfff865d5a366877270930904562e5fd921649c87990c5bb04b33e04e5c521bb" exitCode=0
Sep 16 18:34:28.137986 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.137863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w" event={"ID":"5dccfa63-2b22-4d4f-9793-12f5a021a3ed","Type":"ContainerDied","Data":"dfff865d5a366877270930904562e5fd921649c87990c5bb04b33e04e5c521bb"}
Sep 16 18:34:28.137986 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.137902    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w" event={"ID":"5dccfa63-2b22-4d4f-9793-12f5a021a3ed","Type":"ContainerStarted","Data":"a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943"}
Sep 16 18:34:28.307452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:28.307322    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"]
Sep 16 18:34:28.317815 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:34:28.317788    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1fae95d_40e1_4493_88ac_f8c0d286197c.slice/crio-e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d WatchSource:0}: Error finding container e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d: Status 404 returned error can't find the container with id e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d
Sep 16 18:34:29.145117 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.145038    2576 generic.go:356] "Generic (PLEG): container finished" podID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerID="3be29da6bd9d9d03549ba7770a541de40820d0c3ae32bd265fa99cfc078def0d" exitCode=0
Sep 16 18:34:29.145522 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.145130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4" event={"ID":"cd44afc0-2f78-4e17-8530-5b2d73d4e960","Type":"ContainerDied","Data":"3be29da6bd9d9d03549ba7770a541de40820d0c3ae32bd265fa99cfc078def0d"}
Sep 16 18:34:29.146795 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.146733    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerID="e4710158c216c155ecdfce18e859809a8a8cf562b1afed481efbfad23a3373c9" exitCode=0
Sep 16 18:34:29.146870 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.146798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerDied","Data":"e4710158c216c155ecdfce18e859809a8a8cf562b1afed481efbfad23a3373c9"}
Sep 16 18:34:29.148546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.148528    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerID="c19958eaffec2e72353baaa5ba11b051115e8a25a410282e809edff7e1ac839a" exitCode=0
Sep 16 18:34:29.148632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.148607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf" event={"ID":"a1fae95d-40e1-4493-88ac-f8c0d286197c","Type":"ContainerDied","Data":"c19958eaffec2e72353baaa5ba11b051115e8a25a410282e809edff7e1ac839a"}
Sep 16 18:34:29.148677 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.148645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf" event={"ID":"a1fae95d-40e1-4493-88ac-f8c0d286197c","Type":"ContainerStarted","Data":"e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d"}
Sep 16 18:34:29.150972 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.150948    2576 generic.go:356] "Generic (PLEG): container finished" podID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerID="5c82dad6c496e160e9e92bd0d9dd18e936b583156f2f510e41b72a56ec1b031b" exitCode=0
Sep 16 18:34:29.151028 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:29.150974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w" event={"ID":"5dccfa63-2b22-4d4f-9793-12f5a021a3ed","Type":"ContainerDied","Data":"5c82dad6c496e160e9e92bd0d9dd18e936b583156f2f510e41b72a56ec1b031b"}
Sep 16 18:34:30.159479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:30.159449    2576 generic.go:356] "Generic (PLEG): container finished" podID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerID="6578220f9846f444709d0ab08ea910f744a4ecad4159912c3a6d84a17fe364e5" exitCode=0
Sep 16 18:34:30.159845 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:30.159538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4" event={"ID":"cd44afc0-2f78-4e17-8530-5b2d73d4e960","Type":"ContainerDied","Data":"6578220f9846f444709d0ab08ea910f744a4ecad4159912c3a6d84a17fe364e5"}
Sep 16 18:34:30.161911 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:30.161887    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerStarted","Data":"eda2b887fda5079330a58a2b9a1704cbc34cf5754ed898803f1d9b2558ece038"}
Sep 16 18:34:30.164544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:30.164397    2576 generic.go:356] "Generic (PLEG): container finished" podID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerID="324a6a8e46d44723d5b5af413aa227d92af76ce59680f0da5c5c420fb8699dfb" exitCode=0
Sep 16 18:34:30.164544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:30.164443    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w" event={"ID":"5dccfa63-2b22-4d4f-9793-12f5a021a3ed","Type":"ContainerDied","Data":"324a6a8e46d44723d5b5af413aa227d92af76ce59680f0da5c5c420fb8699dfb"}
Sep 16 18:34:31.171610 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.171573    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerID="eda2b887fda5079330a58a2b9a1704cbc34cf5754ed898803f1d9b2558ece038" exitCode=0
Sep 16 18:34:31.172062 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.171647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerDied","Data":"eda2b887fda5079330a58a2b9a1704cbc34cf5754ed898803f1d9b2558ece038"}
Sep 16 18:34:31.173701 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.173679    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerID="e27acc9558b14e1d2729b9c5abd0ce1a461c9ba8c67320f59e4b1b88a0135b0d" exitCode=0
Sep 16 18:34:31.173775 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.173752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf" event={"ID":"a1fae95d-40e1-4493-88ac-f8c0d286197c","Type":"ContainerDied","Data":"e27acc9558b14e1d2729b9c5abd0ce1a461c9ba8c67320f59e4b1b88a0135b0d"}
Sep 16 18:34:31.470293 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.470245    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:31.473352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.473331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:31.594015 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.593991    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util\") pod \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") "
Sep 16 18:34:31.594152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594040    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjh92\" (UniqueName: \"kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92\") pod \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") "
Sep 16 18:34:31.594152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle\") pod \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") "
Sep 16 18:34:31.594152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594114    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqwbr\" (UniqueName: \"kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr\") pod \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") "
Sep 16 18:34:31.594311 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle\") pod \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\" (UID: \"5dccfa63-2b22-4d4f-9793-12f5a021a3ed\") "
Sep 16 18:34:31.594311 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594196    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util\") pod \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\" (UID: \"cd44afc0-2f78-4e17-8530-5b2d73d4e960\") "
Sep 16 18:34:31.594680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle" (OuterVolumeSpecName: "bundle") pod "5dccfa63-2b22-4d4f-9793-12f5a021a3ed" (UID: "5dccfa63-2b22-4d4f-9793-12f5a021a3ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:31.594680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.594664    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle" (OuterVolumeSpecName: "bundle") pod "cd44afc0-2f78-4e17-8530-5b2d73d4e960" (UID: "cd44afc0-2f78-4e17-8530-5b2d73d4e960"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:31.596373 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.596350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92" (OuterVolumeSpecName: "kube-api-access-gjh92") pod "5dccfa63-2b22-4d4f-9793-12f5a021a3ed" (UID: "5dccfa63-2b22-4d4f-9793-12f5a021a3ed"). InnerVolumeSpecName "kube-api-access-gjh92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:31.596468 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.596438    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr" (OuterVolumeSpecName: "kube-api-access-cqwbr") pod "cd44afc0-2f78-4e17-8530-5b2d73d4e960" (UID: "cd44afc0-2f78-4e17-8530-5b2d73d4e960"). InnerVolumeSpecName "kube-api-access-cqwbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:31.602755 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.602714    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util" (OuterVolumeSpecName: "util") pod "cd44afc0-2f78-4e17-8530-5b2d73d4e960" (UID: "cd44afc0-2f78-4e17-8530-5b2d73d4e960"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:31.603127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.603111    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util" (OuterVolumeSpecName: "util") pod "5dccfa63-2b22-4d4f-9793-12f5a021a3ed" (UID: "5dccfa63-2b22-4d4f-9793-12f5a021a3ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:31.695431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695404    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:31.695431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695429    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:31.695572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695442    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:31.695572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695454    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjh92\" (UniqueName: \"kubernetes.io/projected/5dccfa63-2b22-4d4f-9793-12f5a021a3ed-kube-api-access-gjh92\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:31.695572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695466    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd44afc0-2f78-4e17-8530-5b2d73d4e960-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:31.695572 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:31.695479    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqwbr\" (UniqueName: \"kubernetes.io/projected/cd44afc0-2f78-4e17-8530-5b2d73d4e960-kube-api-access-cqwbr\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:32.181195 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.181164    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4"
Sep 16 18:34:32.181195 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.181166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hjxc4" event={"ID":"cd44afc0-2f78-4e17-8530-5b2d73d4e960","Type":"ContainerDied","Data":"b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94"}
Sep 16 18:34:32.181615 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.181217    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5379bda39ee24818e42a4914c734b5ada3756d443ef7bfbbb41b3a364ff4c94"
Sep 16 18:34:32.183641 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.183614    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerID="5d30ffdee243ff5b7c9e66a2e35fd958c92c794c45e7463b0334dd0d1986540a" exitCode=0
Sep 16 18:34:32.183759 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.183685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerDied","Data":"5d30ffdee243ff5b7c9e66a2e35fd958c92c794c45e7463b0334dd0d1986540a"}
Sep 16 18:34:32.186160 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.186136    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerID="bb2a4367651249956fa4dc17f064da6b686be4dd80f7ce8b3ae49301e767f14b" exitCode=0
Sep 16 18:34:32.186236 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.186209    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf" event={"ID":"a1fae95d-40e1-4493-88ac-f8c0d286197c","Type":"ContainerDied","Data":"bb2a4367651249956fa4dc17f064da6b686be4dd80f7ce8b3ae49301e767f14b"}
Sep 16 18:34:32.188246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.188227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w" event={"ID":"5dccfa63-2b22-4d4f-9793-12f5a021a3ed","Type":"ContainerDied","Data":"a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943"}
Sep 16 18:34:32.188332 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.188262    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2cc5196b7127744811ee07b344b41f044106895fbdfd8a538fea7ee2525a943"
Sep 16 18:34:32.188332 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:32.188232    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jc97w"
Sep 16 18:34:33.330354 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.330331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:33.353086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.353068    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:33.409060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle\") pod \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") "
Sep 16 18:34:33.409060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409059    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util\") pod \"a1fae95d-40e1-4493-88ac-f8c0d286197c\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") "
Sep 16 18:34:33.409228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409081    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjg54\" (UniqueName: \"kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54\") pod \"a1fae95d-40e1-4493-88ac-f8c0d286197c\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") "
Sep 16 18:34:33.409228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle\") pod \"a1fae95d-40e1-4493-88ac-f8c0d286197c\" (UID: \"a1fae95d-40e1-4493-88ac-f8c0d286197c\") "
Sep 16 18:34:33.409228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409155    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util\") pod \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") "
Sep 16 18:34:33.409228 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409191    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgxjm\" (UniqueName: \"kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm\") pod \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\" (UID: \"b7e68d3c-4c06-498e-8efe-d38b9a323c3b\") "
Sep 16 18:34:33.409715 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409637    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle" (OuterVolumeSpecName: "bundle") pod "b7e68d3c-4c06-498e-8efe-d38b9a323c3b" (UID: "b7e68d3c-4c06-498e-8efe-d38b9a323c3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:33.409894 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409861    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:33.410017 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.409952    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle" (OuterVolumeSpecName: "bundle") pod "a1fae95d-40e1-4493-88ac-f8c0d286197c" (UID: "a1fae95d-40e1-4493-88ac-f8c0d286197c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:33.411602 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.411579    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm" (OuterVolumeSpecName: "kube-api-access-lgxjm") pod "b7e68d3c-4c06-498e-8efe-d38b9a323c3b" (UID: "b7e68d3c-4c06-498e-8efe-d38b9a323c3b"). InnerVolumeSpecName "kube-api-access-lgxjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:33.411698 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.411679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54" (OuterVolumeSpecName: "kube-api-access-xjg54") pod "a1fae95d-40e1-4493-88ac-f8c0d286197c" (UID: "a1fae95d-40e1-4493-88ac-f8c0d286197c"). InnerVolumeSpecName "kube-api-access-xjg54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:33.415104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.415072    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util" (OuterVolumeSpecName: "util") pod "b7e68d3c-4c06-498e-8efe-d38b9a323c3b" (UID: "b7e68d3c-4c06-498e-8efe-d38b9a323c3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:33.415792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.415772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util" (OuterVolumeSpecName: "util") pod "a1fae95d-40e1-4493-88ac-f8c0d286197c" (UID: "a1fae95d-40e1-4493-88ac-f8c0d286197c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:33.511072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.511020    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjg54\" (UniqueName: \"kubernetes.io/projected/a1fae95d-40e1-4493-88ac-f8c0d286197c-kube-api-access-xjg54\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:33.511072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.511040    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:33.511072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.511050    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:33.511072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.511058    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgxjm\" (UniqueName: \"kubernetes.io/projected/b7e68d3c-4c06-498e-8efe-d38b9a323c3b-kube-api-access-lgxjm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:33.511072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:33.511067    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1fae95d-40e1-4493-88ac-f8c0d286197c-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:34:34.201882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.201845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc" event={"ID":"b7e68d3c-4c06-498e-8efe-d38b9a323c3b","Type":"ContainerDied","Data":"c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff"}
Sep 16 18:34:34.201882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.201878    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmvzsc"
Sep 16 18:34:34.202130 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.201900    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8df481327f6d2f8d4f5db6dc04cef86da86343948a2d4964fc520ce8cfee6ff"
Sep 16 18:34:34.203923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.203894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf" event={"ID":"a1fae95d-40e1-4493-88ac-f8c0d286197c","Type":"ContainerDied","Data":"e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d"}
Sep 16 18:34:34.204057 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.203913    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8wplf"
Sep 16 18:34:34.204057 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:34.203962    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3945a7f28ecd6b803c5d57407aff0c841b7345b723f9316d9067fb7bbd3853d"
Sep 16 18:34:46.575028 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.574988    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796498f4c7-qpd8g"]
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575641    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575658    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575665    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575670    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575681    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575687    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575694    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575699    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575708    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575713    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575723    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575728    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575733    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575746    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575751    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575762    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575767    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575774    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575780    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575800    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575805    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="util"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575811    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575816    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="pull"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575871    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd44afc0-2f78-4e17-8530-5b2d73d4e960" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575881    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dccfa63-2b22-4d4f-9793-12f5a021a3ed" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575887    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e68d3c-4c06-498e-8efe-d38b9a323c3b" containerName="extract"
Sep 16 18:34:46.578897 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.575893    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1fae95d-40e1-4493-88ac-f8c0d286197c" containerName="extract"
Sep 16 18:34:46.580981 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.580962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.621692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.621658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796498f4c7-qpd8g"]
Sep 16 18:34:46.724798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.724967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-trusted-ca-bundle\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.724967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-oauth-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.724967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.724967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-service-ca\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.724967 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.724962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjlb\" (UniqueName: \"kubernetes.io/projected/cd762a9f-a7ea-48d9-9645-42bce6b568db-kube-api-access-4tjlb\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.725147 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.725026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-oauth-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826067 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-oauth-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826067 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826067 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-service-ca\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjlb\" (UniqueName: \"kubernetes.io/projected/cd762a9f-a7ea-48d9-9645-42bce6b568db-kube-api-access-4tjlb\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-oauth-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826243 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-trusted-ca-bundle\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826797 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-service-ca\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-oauth-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.826913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.826853    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.827407 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.827386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd762a9f-a7ea-48d9-9645-42bce6b568db-trusted-ca-bundle\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.828832 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.828809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-serving-cert\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.828924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.828818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd762a9f-a7ea-48d9-9645-42bce6b568db-console-oauth-config\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:46.890821 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:46.890792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjlb\" (UniqueName: \"kubernetes.io/projected/cd762a9f-a7ea-48d9-9645-42bce6b568db-kube-api-access-4tjlb\") pod \"console-796498f4c7-qpd8g\" (UID: \"cd762a9f-a7ea-48d9-9645-42bce6b568db\") " pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:47.191193 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:47.191162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:47.614950 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:47.614841    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796498f4c7-qpd8g"]
Sep 16 18:34:47.619672 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:34:47.619639    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd762a9f_a7ea_48d9_9645_42bce6b568db.slice/crio-44c845b13181c77116b3528688e3ba861e49005c8e1841b1f5672329f2d0611b WatchSource:0}: Error finding container 44c845b13181c77116b3528688e3ba861e49005c8e1841b1f5672329f2d0611b: Status 404 returned error can't find the container with id 44c845b13181c77116b3528688e3ba861e49005c8e1841b1f5672329f2d0611b
Sep 16 18:34:48.283774 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:48.283737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796498f4c7-qpd8g" event={"ID":"cd762a9f-a7ea-48d9-9645-42bce6b568db","Type":"ContainerStarted","Data":"ffed511e93c3aee9f6ae591275d84abde52faa179da406c11c39a58d6d9c1563"}
Sep 16 18:34:48.283970 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:48.283796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796498f4c7-qpd8g" event={"ID":"cd762a9f-a7ea-48d9-9645-42bce6b568db","Type":"ContainerStarted","Data":"44c845b13181c77116b3528688e3ba861e49005c8e1841b1f5672329f2d0611b"}
Sep 16 18:34:48.336587 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:48.336540    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796498f4c7-qpd8g" podStartSLOduration=2.336523444 podStartE2EDuration="2.336523444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:48.333346374 +0000 UTC m=+415.753833235" watchObservedRunningTime="2026-09-16 18:34:48.336523444 +0000 UTC m=+415.757010377"
Sep 16 18:34:57.192350 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:57.192317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:57.192748 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:57.192377    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:57.196945 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:57.196907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:57.337664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:57.337638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-796498f4c7-qpd8g"
Sep 16 18:34:57.450123 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:34:57.450050    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:35:22.469346 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.469241    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79478bcdbc-b24cc" podUID="597cdfa0-ea8a-4732-a031-f7887d14fef9" containerName="console" containerID="cri-o://f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71" gracePeriod=15
Sep 16 18:35:22.715639 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.715618    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:35:22.822372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822311    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822342    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822373    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822592 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822391    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822592 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822448    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822592 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9hlx\" (UniqueName: \"kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.822592 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822482    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert\") pod \"597cdfa0-ea8a-4732-a031-f7887d14fef9\" (UID: \"597cdfa0-ea8a-4732-a031-f7887d14fef9\") "
Sep 16 18:35:22.823059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822812    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:22.823059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822862    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config" (OuterVolumeSpecName: "console-config") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:22.823059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.822996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca" (OuterVolumeSpecName: "service-ca") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:22.823059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.823013    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:22.824765 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.824739    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx" (OuterVolumeSpecName: "kube-api-access-h9hlx") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "kube-api-access-h9hlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:22.825171 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.825153    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:35:22.825262 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.825171    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "597cdfa0-ea8a-4732-a031-f7887d14fef9" (UID: "597cdfa0-ea8a-4732-a031-f7887d14fef9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:35:22.923992 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.923973    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.923992 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.923992    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9hlx\" (UniqueName: \"kubernetes.io/projected/597cdfa0-ea8a-4732-a031-f7887d14fef9-kube-api-access-h9hlx\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.924151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.924002    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.924151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.924011    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.924151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.924019    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.924151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.924026    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:22.924151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:22.924034    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/597cdfa0-ea8a-4732-a031-f7887d14fef9-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:35:23.481198 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.481164    2576 generic.go:356] "Generic (PLEG): container finished" podID="597cdfa0-ea8a-4732-a031-f7887d14fef9" containerID="f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71" exitCode=0
Sep 16 18:35:23.481582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.481234    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79478bcdbc-b24cc"
Sep 16 18:35:23.481582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.481251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79478bcdbc-b24cc" event={"ID":"597cdfa0-ea8a-4732-a031-f7887d14fef9","Type":"ContainerDied","Data":"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"}
Sep 16 18:35:23.481582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.481327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79478bcdbc-b24cc" event={"ID":"597cdfa0-ea8a-4732-a031-f7887d14fef9","Type":"ContainerDied","Data":"fa57219dd9c844d977b397f05b247c93fde95935a506bee2c6a70fc9c4eac00d"}
Sep 16 18:35:23.481582 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.481347    2576 scope.go:117] "RemoveContainer" containerID="f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"
Sep 16 18:35:23.490549 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.490531    2576 scope.go:117] "RemoveContainer" containerID="f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"
Sep 16 18:35:23.490819 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:35:23.490795    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71\": container with ID starting with f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71 not found: ID does not exist" containerID="f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"
Sep 16 18:35:23.490898 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.490828    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71"} err="failed to get container status \"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71\": rpc error: code = NotFound desc = could not find container \"f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71\": container with ID starting with f475cc66218d0af562b0b8b8d52a0cda5eac099efecd3d7b4b38ce98485b7a71 not found: ID does not exist"
Sep 16 18:35:23.512963 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.512919    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:35:23.525771 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:23.525742    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79478bcdbc-b24cc"]
Sep 16 18:35:25.010473 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:35:25.010442    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="597cdfa0-ea8a-4732-a031-f7887d14fef9" path="/var/lib/kubelet/pods/597cdfa0-ea8a-4732-a031-f7887d14fef9/volumes"
Sep 16 18:36:25.692579 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.692550    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:25.693105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.692924    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597cdfa0-ea8a-4732-a031-f7887d14fef9" containerName="console"
Sep 16 18:36:25.693105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.692950    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="597cdfa0-ea8a-4732-a031-f7887d14fef9" containerName="console"
Sep 16 18:36:25.693105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.693005    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="597cdfa0-ea8a-4732-a031-f7887d14fef9" containerName="console"
Sep 16 18:36:25.696003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.695989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.698623 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.698601    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 18:36:25.698849 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.698836    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 18:36:25.699721 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.699705    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cgvng\""
Sep 16 18:36:25.699915 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.699900    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 18:36:25.712570 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.712548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:25.815349 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.815327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.815469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.815357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4tv7\" (UniqueName: \"kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.815469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.815461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.915993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.915956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.915993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.915999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4tv7\" (UniqueName: \"kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.916144 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.916055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.916181 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:36:25.916154    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 16 18:36:25.916215 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:36:25.916206    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert podName:6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8 nodeName:}" failed. No retries permitted until 2026-09-16 18:36:26.416190637 +0000 UTC m=+513.836677484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert") pod "maas-controller-75b7845b9b-5rxp2" (UID: "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8") : secret "maas-controller-webhook-cert" not found
Sep 16 18:36:25.918650 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.918629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.929818 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.929794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4tv7\" (UniqueName: \"kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:25.967673 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.967620    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 18:36:25.971003 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.970989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:25.990629 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:25.990608    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 18:36:26.117808 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.117781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl85b\" (UniqueName: \"kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.117971 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.117816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.118048 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.117997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.219122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.219060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.219122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.219121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl85b\" (UniqueName: \"kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.219297 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.219144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.221675 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.221653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.221783 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.221691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.229031 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.228998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl85b\" (UniqueName: \"kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b\") pod \"maas-controller-8d84d5c46-lvkn2\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") " pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.281004 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.280980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:26.420858 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.420830    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:26.423281 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.423260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") pod \"maas-controller-75b7845b9b-5rxp2\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") " pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:26.587298 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.587274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 18:36:26.588447 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:36:26.588414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca49a7f_8c40_4076_bd42_7f3a21892c9a.slice/crio-bc241650b7a4c58a7dffaea6467b66b4514b1d501d018321480ac366721e9901 WatchSource:0}: Error finding container bc241650b7a4c58a7dffaea6467b66b4514b1d501d018321480ac366721e9901: Status 404 returned error can't find the container with id bc241650b7a4c58a7dffaea6467b66b4514b1d501d018321480ac366721e9901
Sep 16 18:36:26.606950 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.606909    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:26.833395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.833362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" event={"ID":"2ca49a7f-8c40-4076-bd42-7f3a21892c9a","Type":"ContainerStarted","Data":"bc241650b7a4c58a7dffaea6467b66b4514b1d501d018321480ac366721e9901"}
Sep 16 18:36:26.890355 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:26.890335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:27.843179 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:27.843106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" event={"ID":"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8","Type":"ContainerStarted","Data":"4f9e1829356d42d128ed31f63fba55e2eaf8bb6ea746e908603e2581405537f8"}
Sep 16 18:36:29.863229 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.863192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" event={"ID":"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8","Type":"ContainerStarted","Data":"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"}
Sep 16 18:36:29.863705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.863306    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:29.864916 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.864884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" event={"ID":"2ca49a7f-8c40-4076-bd42-7f3a21892c9a","Type":"ContainerStarted","Data":"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"}
Sep 16 18:36:29.865062 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.865043    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:29.914594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.914531    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" podStartSLOduration=2.581067709 podStartE2EDuration="4.914516612s" totalImagesPullingTime="2.333448903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:29.907783362 +0000 UTC m=+517.328270223" watchObservedRunningTime="2026-09-16 18:36:29.914516612 +0000 UTC m=+517.335003472"
Sep 16 18:36:29.992803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:29.992749    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" podStartSLOduration=2.361086742 podStartE2EDuration="4.992734388s" totalImagesPullingTime="2.631647646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:29.98353058 +0000 UTC m=+517.404017440" watchObservedRunningTime="2026-09-16 18:36:29.992734388 +0000 UTC m=+517.413221248"
Sep 16 18:36:40.872689 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:40.872657    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:40.873193 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:40.872741    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 18:36:40.975085 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:40.975054    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:40.975311 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:40.975240    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" podUID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" containerName="manager" containerID="cri-o://1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca" gracePeriod=30
Sep 16 18:36:41.215634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.215612    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:41.233444 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.233419    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4tv7\" (UniqueName: \"kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7\") pod \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") "
Sep 16 18:36:41.233568 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.233479    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") pod \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") "
Sep 16 18:36:41.233568 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.233519    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs\") pod \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\" (UID: \"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8\") "
Sep 16 18:36:41.235672 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.235645    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7" (OuterVolumeSpecName: "kube-api-access-k4tv7") pod "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" (UID: "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8"). InnerVolumeSpecName "kube-api-access-k4tv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:41.235885 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.235856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" (UID: "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:36:41.236037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.235885    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" (UID: "6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:36:41.334104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.334070    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4tv7\" (UniqueName: \"kubernetes.io/projected/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-kube-api-access-k4tv7\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:36:41.334104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.334098    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-webhook-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:36:41.334104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.334110    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:36:41.934762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.934726    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" containerID="1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca" exitCode=0
Sep 16 18:36:41.935211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.934771    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75b7845b9b-5rxp2"
Sep 16 18:36:41.935211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.934804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" event={"ID":"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8","Type":"ContainerDied","Data":"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"}
Sep 16 18:36:41.935211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.934871    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75b7845b9b-5rxp2" event={"ID":"6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8","Type":"ContainerDied","Data":"4f9e1829356d42d128ed31f63fba55e2eaf8bb6ea746e908603e2581405537f8"}
Sep 16 18:36:41.935211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.934894    2576 scope.go:117] "RemoveContainer" containerID="1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"
Sep 16 18:36:41.945027 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.945009    2576 scope.go:117] "RemoveContainer" containerID="1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"
Sep 16 18:36:41.945324 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:36:41.945306    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca\": container with ID starting with 1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca not found: ID does not exist" containerID="1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"
Sep 16 18:36:41.945370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.945334    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca"} err="failed to get container status \"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca\": rpc error: code = NotFound desc = could not find container \"1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca\": container with ID starting with 1930c35372c378d8412d45560de998ace660270a6f0c8d9abc44c1b4b23f04ca not found: ID does not exist"
Sep 16 18:36:41.960364 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.960339    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:41.967640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:41.967620    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75b7845b9b-5rxp2"]
Sep 16 18:36:43.009991 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:36:43.009957    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" path="/var/lib/kubelet/pods/6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8/volumes"
Sep 16 18:37:03.725966 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.725918    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"]
Sep 16 18:37:03.726455 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.726308    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" containerName="manager"
Sep 16 18:37:03.726455 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.726319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" containerName="manager"
Sep 16 18:37:03.726455 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.726403    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ab05740-0c5b-40b8-99e3-bc9ccf03b6b8" containerName="manager"
Sep 16 18:37:03.734118 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.734099    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.736608 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.736585    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 18:37:03.736608 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.736600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cvqhq\""
Sep 16 18:37:03.746153 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.746124    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"]
Sep 16 18:37:03.815339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.815308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9m6r\" (UniqueName: \"kubernetes.io/projected/d485e4b5-5877-44d5-9e35-767ee27bb96f-kube-api-access-f9m6r\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.815479 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.815351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d485e4b5-5877-44d5-9e35-767ee27bb96f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.916450 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.916418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9m6r\" (UniqueName: \"kubernetes.io/projected/d485e4b5-5877-44d5-9e35-767ee27bb96f-kube-api-access-f9m6r\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.916613 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.916463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d485e4b5-5877-44d5-9e35-767ee27bb96f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.917167 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.917149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d485e4b5-5877-44d5-9e35-767ee27bb96f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:03.926678 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:03.926647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9m6r\" (UniqueName: \"kubernetes.io/projected/d485e4b5-5877-44d5-9e35-767ee27bb96f-kube-api-access-f9m6r\") pod \"payload-processing-5b54b7fbb7-8cjcd\" (UID: \"d485e4b5-5877-44d5-9e35-767ee27bb96f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:04.051469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:04.051386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:04.332849 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:04.332817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"]
Sep 16 18:37:04.334071 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:37:04.334048    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd485e4b5_5877_44d5_9e35_767ee27bb96f.slice/crio-2758d720a43f1190851b934595cb492d9fff98c0c38f65718d4e96d57bcece09 WatchSource:0}: Error finding container 2758d720a43f1190851b934595cb492d9fff98c0c38f65718d4e96d57bcece09: Status 404 returned error can't find the container with id 2758d720a43f1190851b934595cb492d9fff98c0c38f65718d4e96d57bcece09
Sep 16 18:37:05.069536 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:05.069499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd" event={"ID":"d485e4b5-5877-44d5-9e35-767ee27bb96f","Type":"ContainerStarted","Data":"2758d720a43f1190851b934595cb492d9fff98c0c38f65718d4e96d57bcece09"}
Sep 16 18:37:06.078255 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:06.078192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd" event={"ID":"d485e4b5-5877-44d5-9e35-767ee27bb96f","Type":"ContainerStarted","Data":"68523f68350b0dea73cd02a7113192feb5d32ccb76495e994f112e4d5f0b1eb9"}
Sep 16 18:37:06.078641 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:06.078376    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:06.101670 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:06.101621    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd" podStartSLOduration=1.446168481 podStartE2EDuration="3.101603967s" totalImagesPullingTime="1.655435486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:06.100570144 +0000 UTC m=+553.521057007" watchObservedRunningTime="2026-09-16 18:37:06.101603967 +0000 UTC m=+553.522090827"
Sep 16 18:37:17.086602 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:17.086574    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8cjcd"
Sep 16 18:37:25.151456 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.151424    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"]
Sep 16 18:37:25.154827 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.154807    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.157552 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.157529    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 18:37:25.157664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.157625    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 18:37:25.158661 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.158645    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pg6k\""
Sep 16 18:37:25.158735 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.158695    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 18:37:25.169826 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.169801    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"]
Sep 16 18:37:25.305882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.305848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.305882 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.305885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.306110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.305967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.306110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.306001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.306110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.306066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.306110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.306105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtmrq\" (UniqueName: \"kubernetes.io/projected/af44e0d9-a5f6-44a8-930f-2d06417e52b6-kube-api-access-qtmrq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtmrq\" (UniqueName: \"kubernetes.io/projected/af44e0d9-a5f6-44a8-930f-2d06417e52b6-kube-api-access-qtmrq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407296 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407296 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407296 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407296 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407600 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.407660 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.407604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.409514 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.409495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af44e0d9-a5f6-44a8-930f-2d06417e52b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.409785 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.409767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af44e0d9-a5f6-44a8-930f-2d06417e52b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.419909 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.419889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtmrq\" (UniqueName: \"kubernetes.io/projected/af44e0d9-a5f6-44a8-930f-2d06417e52b6-kube-api-access-qtmrq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84\" (UID: \"af44e0d9-a5f6-44a8-930f-2d06417e52b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.465594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.465566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:25.768328 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:37:25.768296    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf44e0d9_a5f6_44a8_930f_2d06417e52b6.slice/crio-500679c1c655fa4a7a3bda22d5903edf4f427c47edd0aa063ecf5fe05fb88ba6 WatchSource:0}: Error finding container 500679c1c655fa4a7a3bda22d5903edf4f427c47edd0aa063ecf5fe05fb88ba6: Status 404 returned error can't find the container with id 500679c1c655fa4a7a3bda22d5903edf4f427c47edd0aa063ecf5fe05fb88ba6
Sep 16 18:37:25.782705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:25.782676    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"]
Sep 16 18:37:26.189298 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:26.189256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84" event={"ID":"af44e0d9-a5f6-44a8-930f-2d06417e52b6","Type":"ContainerStarted","Data":"500679c1c655fa4a7a3bda22d5903edf4f427c47edd0aa063ecf5fe05fb88ba6"}
Sep 16 18:37:27.196924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:27.196891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84" event={"ID":"af44e0d9-a5f6-44a8-930f-2d06417e52b6","Type":"ContainerStarted","Data":"03b9770fd2e2593bccf758f6e2ff8f14146ceb0c37e8e3d92620c25c06efe627"}
Sep 16 18:37:35.466478 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:35.466441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:35.466964 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:35.466505    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:35.479059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:35.479036    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:37:35.523455 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:35.523406    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84" podStartSLOduration=9.541502386 podStartE2EDuration="10.523392951s" totalImagesPullingTime="981.890565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:27.251356822 +0000 UTC m=+574.671843682" watchObservedRunningTime="2026-09-16 18:37:35.523392951 +0000 UTC m=+582.943879809"
Sep 16 18:37:36.255824 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:37:36.255801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84"
Sep 16 18:40:37.780968 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.780921    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:40:37.784526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.784500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.787315 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.787287    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4dda98-openshift-default-dockercfg-hnpgz\""
Sep 16 18:40:37.803113 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.803090    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:40:37.923452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923452 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl27x\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:37.923790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:37.923784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.024893 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.024859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025089 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.024907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl27x\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025089 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.024951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025089 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025474 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025474 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025474 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025586 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025497    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025586 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.025779 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.025760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.027564 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.027539    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.027741 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.027722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.038923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.038862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl27x\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.039105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.039088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token\") pod \"e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") " pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.096082 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.096057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:38.385619 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.385598    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:40:38.387732 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:38.387701    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42734aa2_125f_4cfe_847c_44d413960998.slice/crio-5abfea37d6f048d97d1c49026faef4cd34b94db1feb078464f871dc7bf0e4728 WatchSource:0}: Error finding container 5abfea37d6f048d97d1c49026faef4cd34b94db1feb078464f871dc7bf0e4728: Status 404 returned error can't find the container with id 5abfea37d6f048d97d1c49026faef4cd34b94db1feb078464f871dc7bf0e4728
Sep 16 18:40:38.389919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:38.389901    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:40:39.265703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:39.265672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerStarted","Data":"5abfea37d6f048d97d1c49026faef4cd34b94db1feb078464f871dc7bf0e4728"}
Sep 16 18:40:41.679851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.679817    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:41.680175 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.679889    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:41.680175 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.679918    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:41.919894 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.919865    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:40:41.923642 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.923623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:41.926860 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.926833    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6f41de-serving-cert\""
Sep 16 18:40:41.927895 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.927827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 18:40:41.928051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.927962    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:40:41.928193 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.928118    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:40:41.928311 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.928293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lpqld\""
Sep 16 18:40:41.943549 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:41.943524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:40:42.066142 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.066103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.066142 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.066139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.066335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.066214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkvg2\" (UniqueName: \"kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.066423 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.066400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.167713 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.167679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.167713 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.167715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.167969 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.167733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.167969 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.167760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkvg2\" (UniqueName: \"kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.168323 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.168288    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.170294 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.170269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.170425 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.170395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.178448 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.178386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkvg2\" (UniqueName: \"kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2\") pod \"maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.236247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.236224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:42.289702 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.289666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerStarted","Data":"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"}
Sep 16 18:40:42.322598 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.322548    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podStartSLOduration=2.033104502 podStartE2EDuration="5.322532068s" totalImagesPullingTime="3.289427566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:42.320705263 +0000 UTC m=+769.741192123" watchObservedRunningTime="2026-09-16 18:40:42.322532068 +0000 UTC m=+769.743018928"
Sep 16 18:40:42.528744 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:42.528706    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:40:42.530250 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:42.530215    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a6d3ca_21d8_4fe0_8df6_0c87bd19026e.slice/crio-8132da5e403e1b65384a6b49d4968ea91d33d9e04b889f5d6edd739a6b97c126 WatchSource:0}: Error finding container 8132da5e403e1b65384a6b49d4968ea91d33d9e04b889f5d6edd739a6b97c126: Status 404 returned error can't find the container with id 8132da5e403e1b65384a6b49d4968ea91d33d9e04b889f5d6edd739a6b97c126
Sep 16 18:40:43.097090 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.097060    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:43.103695 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.103673    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:43.187875 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.187836    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:40:43.192278 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.192255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.194912 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.194882    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6f41de\""
Sep 16 18:40:43.195094 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.195070    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6f41de-dockercfg-v429m\""
Sep 16 18:40:43.220060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.220029    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:40:43.285329 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.284948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2rn\" (UniqueName: \"kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.285329 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.285110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.302079 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.302041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" event={"ID":"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e","Type":"ContainerStarted","Data":"8132da5e403e1b65384a6b49d4968ea91d33d9e04b889f5d6edd739a6b97c126"}
Sep 16 18:40:43.302483 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.302448    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:43.303401 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.303374    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:40:43.386117 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.385981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.386117 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.386073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2rn\" (UniqueName: \"kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.387100 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.387065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.400136 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.400110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2rn\" (UniqueName: \"kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn\") pod \"payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.508194 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.508162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:43.802301 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:43.802212    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:40:43.804603 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:43.804573    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ae63d5_3ba9_406e_9bea_ef5c478e58b0.slice/crio-ad5cd8bd410bd9ef3ef4804651b81240d6ac34df473e7058bc388502429088e7 WatchSource:0}: Error finding container ad5cd8bd410bd9ef3ef4804651b81240d6ac34df473e7058bc388502429088e7: Status 404 returned error can't find the container with id ad5cd8bd410bd9ef3ef4804651b81240d6ac34df473e7058bc388502429088e7
Sep 16 18:40:44.310952 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:44.310776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" event={"ID":"47ae63d5-3ba9-406e-9bea-ef5c478e58b0","Type":"ContainerStarted","Data":"ba15c3603879af0c945c98e93934ab46c4d5e1a7ea022c8c919123d55830cf5e"}
Sep 16 18:40:44.310952 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:44.310841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" event={"ID":"47ae63d5-3ba9-406e-9bea-ef5c478e58b0","Type":"ContainerStarted","Data":"ad5cd8bd410bd9ef3ef4804651b81240d6ac34df473e7058bc388502429088e7"}
Sep 16 18:40:44.311409 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:44.311041    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:44.375690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:44.375632    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" podStartSLOduration=1.375613699 podStartE2EDuration="1.375613699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:44.366030934 +0000 UTC m=+771.786517796" watchObservedRunningTime="2026-09-16 18:40:44.375613699 +0000 UTC m=+771.796100591"
Sep 16 18:40:45.319175 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:45.319145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" event={"ID":"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e","Type":"ContainerStarted","Data":"3d639f28e81dfa677e04a44077d7bc59ae91524367bd65e0765007b138a80a79"}
Sep 16 18:40:45.319606 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:45.319469    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:45.366532 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:45.366481    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" podStartSLOduration=2.297768395 podStartE2EDuration="4.366465314s" totalImagesPullingTime="2.068696919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:45.364419005 +0000 UTC m=+772.784905870" watchObservedRunningTime="2026-09-16 18:40:45.366465314 +0000 UTC m=+772.786952174"
Sep 16 18:40:47.516181 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.516148    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:40:47.519747 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.519728    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.522116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.522092    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-4dda98-serving-cert\""
Sep 16 18:40:47.539870 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.539845    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:40:47.625842 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.625807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.626041 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.626009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.626190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.626170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.626276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.626223    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbkql\" (UniqueName: \"kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.727201 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.727170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.727356 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.727222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbkql\" (UniqueName: \"kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.727356 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.727259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.727486 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.727373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.727544 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:47.727535    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-serving-cert: secret "maas-api-e2e-worker-w0-4dda98-serving-cert" not found
Sep 16 18:40:47.727611 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:47.727599    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls podName:5d248cfc-2730-4165-bec4-949bb7d62056 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:48.227581926 +0000 UTC m=+775.648068781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls") pod "maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" (UID: "5d248cfc-2730-4165-bec4-949bb7d62056") : secret "maas-api-e2e-worker-w0-4dda98-serving-cert" not found
Sep 16 18:40:47.727946 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.727905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.730103 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.730084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.752268 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.752248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbkql\" (UniqueName: \"kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:47.949791 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.949758    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:40:47.953713 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.953694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:47.956662 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:47.956643    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-90ea7d-openshift-default-dockercfg-b6t27\""
Sep 16 18:40:48.046481 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.046450    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:40:48.130201 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130377 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130217    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130377 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130377 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkb82\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.130710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.130622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.231896 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.231818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.231896 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.231863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.231907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkb82\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.231925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.231991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232058    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232491 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:48.232491 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232491 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232491 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232690 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.232975 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.232757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.234629 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.234599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.234737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.234651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:48.234737 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.234690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.251876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.251854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.259494 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.259472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkb82\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82\") pod \"e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") " pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.264454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.264421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:48.430568 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.430532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:48.589584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.589559    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:40:48.591824 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:48.591799    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b31970_25a6_4961_82fb_f058ac2eaf58.slice/crio-c6266ae6f96cc69385c804e4f5bbffce51102b9e8927825ac99953b1b20ccdfd WatchSource:0}: Error finding container c6266ae6f96cc69385c804e4f5bbffce51102b9e8927825ac99953b1b20ccdfd: Status 404 returned error can't find the container with id c6266ae6f96cc69385c804e4f5bbffce51102b9e8927825ac99953b1b20ccdfd
Sep 16 18:40:48.595301 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.595267    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:48.595371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.595336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:48.595371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.595365    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:40:48.747453 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:48.747423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:40:48.749058 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:48.749022    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d248cfc_2730_4165_bec4_949bb7d62056.slice/crio-d9bc2765f2370917552e03dd5a6573de1836e8936ed11b1ad3a5279a14fbe9fa WatchSource:0}: Error finding container d9bc2765f2370917552e03dd5a6573de1836e8936ed11b1ad3a5279a14fbe9fa: Status 404 returned error can't find the container with id d9bc2765f2370917552e03dd5a6573de1836e8936ed11b1ad3a5279a14fbe9fa
Sep 16 18:40:49.348662 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.348629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" event={"ID":"5d248cfc-2730-4165-bec4-949bb7d62056","Type":"ContainerStarted","Data":"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"}
Sep 16 18:40:49.348851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.348692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" event={"ID":"5d248cfc-2730-4165-bec4-949bb7d62056","Type":"ContainerStarted","Data":"d9bc2765f2370917552e03dd5a6573de1836e8936ed11b1ad3a5279a14fbe9fa"}
Sep 16 18:40:49.348851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.348799    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:49.350513 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.350488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerStarted","Data":"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"}
Sep 16 18:40:49.350626 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.350534    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerStarted","Data":"c6266ae6f96cc69385c804e4f5bbffce51102b9e8927825ac99953b1b20ccdfd"}
Sep 16 18:40:49.389025 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.388462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" podStartSLOduration=2.03638111 podStartE2EDuration="2.38844364s" totalImagesPullingTime="352.06253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:49.387814395 +0000 UTC m=+776.808301248" watchObservedRunningTime="2026-09-16 18:40:49.38844364 +0000 UTC m=+776.808930504"
Sep 16 18:40:49.432982 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:49.432904    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podStartSLOduration=2.432882246 podStartE2EDuration="2.432882246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:49.430781806 +0000 UTC m=+776.851268678" watchObservedRunningTime="2026-09-16 18:40:49.432882246 +0000 UTC m=+776.853369107"
Sep 16 18:40:50.264970 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:50.264920    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:50.269817 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:50.269795    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:50.357312 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:50.357283    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:50.358096 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:50.358081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:40:51.330348 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:51.330319    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:40:55.321103 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.321075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:40:55.362333 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.362305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:40:55.559973 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.559946    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:40:55.563899 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.563878    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.566622 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.566603    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-90ea7d-serving-cert\""
Sep 16 18:40:55.591113 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.591060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:40:55.603384 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.603362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8js65\" (UniqueName: \"kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.603483 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.603398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.603546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.603528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.603583 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.603563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.704795 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.704766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8js65\" (UniqueName: \"kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.705002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.704803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.705002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.704888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.705002 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.704914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.705181 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:55.705027    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-serving-cert: secret "maas-api-e2e-autores-90ea7d-serving-cert" not found
Sep 16 18:40:55.705181 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:55.705086    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls podName:cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:56.205070991 +0000 UTC m=+783.625557828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls") pod "maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741") : secret "maas-api-e2e-autores-90ea7d-serving-cert" not found
Sep 16 18:40:55.705613 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.705590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.707496 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.707476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:55.723223 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:55.723196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8js65\" (UniqueName: \"kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:56.210669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:56.210636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:56.210831 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:56.210778    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-serving-cert: secret "maas-api-e2e-autores-90ea7d-serving-cert" not found
Sep 16 18:40:56.210889 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:40:56.210842    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls podName:cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:57.210825669 +0000 UTC m=+784.631312506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls") pod "maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741") : secret "maas-api-e2e-autores-90ea7d-serving-cert" not found
Sep 16 18:40:57.221038 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:57.220980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:57.223682 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:57.223656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:57.374371 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:57.374327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:57.671409 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:57.671379    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:40:57.672344 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:40:57.672314    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd6ec2c_a67f_4b40_89b8_3d32e3c6a741.slice/crio-dc6940bee53cf100d3414890c0266007821176793f846958733837ef86783d2c WatchSource:0}: Error finding container dc6940bee53cf100d3414890c0266007821176793f846958733837ef86783d2c: Status 404 returned error can't find the container with id dc6940bee53cf100d3414890c0266007821176793f846958733837ef86783d2c
Sep 16 18:40:58.413387 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:58.413353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" event={"ID":"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741","Type":"ContainerStarted","Data":"dc6940bee53cf100d3414890c0266007821176793f846958733837ef86783d2c"}
Sep 16 18:40:59.421310 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:59.421274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" event={"ID":"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741","Type":"ContainerStarted","Data":"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"}
Sep 16 18:40:59.421756 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:59.421418    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:40:59.442424 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:40:59.442357    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" podStartSLOduration=3.633412151 podStartE2EDuration="4.442338762s" totalImagesPullingTime="808.926611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:59.441620929 +0000 UTC m=+786.862107802" watchObservedRunningTime="2026-09-16 18:40:59.442338762 +0000 UTC m=+786.862825627"
Sep 16 18:41:01.445109 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.445080    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:01.451414 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.451389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.451636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.451608    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:01.454613 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.454593    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b7e002c7-serving-cert\""
Sep 16 18:41:01.566141 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.566103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2m6l\" (UniqueName: \"kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.566302 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.566203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.566302 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.566233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.566373 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.566312    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.666835 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.666800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.666835 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.666838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.667121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.666888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.667121 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.666923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2m6l\" (UniqueName: \"kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.667121 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:41:01.666976    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-serving-cert: secret "maas-api-e2e-ait-b7e002c7-serving-cert" not found
Sep 16 18:41:01.667121 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:41:01.667057    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls podName:c4d1648d-7972-4c1b-a949-61afc6139644 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:02.167034523 +0000 UTC m=+789.587521362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls") pod "maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" (UID: "c4d1648d-7972-4c1b-a949-61afc6139644") : secret "maas-api-e2e-ait-b7e002c7-serving-cert" not found
Sep 16 18:41:01.667668 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.667646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.669449 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.669419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:01.684891 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:01.684869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2m6l\" (UniqueName: \"kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:02.172699 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:02.172661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:02.175216 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:02.175194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") pod \"maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:02.365305 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:02.365275    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:02.664304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:02.664278    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:02.665607 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:41:02.665585    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d1648d_7972_4c1b_a949_61afc6139644.slice/crio-0153a2a08f55152104c736737e6c616ddf1224b376d50f8ee0598bc89e6a39c3 WatchSource:0}: Error finding container 0153a2a08f55152104c736737e6c616ddf1224b376d50f8ee0598bc89e6a39c3: Status 404 returned error can't find the container with id 0153a2a08f55152104c736737e6c616ddf1224b376d50f8ee0598bc89e6a39c3
Sep 16 18:41:03.470010 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:03.469967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" event={"ID":"c4d1648d-7972-4c1b-a949-61afc6139644","Type":"ContainerStarted","Data":"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"}
Sep 16 18:41:03.470198 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:03.470059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" event={"ID":"c4d1648d-7972-4c1b-a949-61afc6139644","Type":"ContainerStarted","Data":"0153a2a08f55152104c736737e6c616ddf1224b376d50f8ee0598bc89e6a39c3"}
Sep 16 18:41:03.470198 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:03.470114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:03.509395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:03.509341    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" podStartSLOduration=2.199747306 podStartE2EDuration="2.509320005s" totalImagesPullingTime="309.572699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:03.507225949 +0000 UTC m=+790.927712811" watchObservedRunningTime="2026-09-16 18:41:03.509320005 +0000 UTC m=+790.929806868"
Sep 16 18:41:05.437781 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:05.437755    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:41:08.504623 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.504587    2576 generic.go:356] "Generic (PLEG): container finished" podID="42734aa2-125f-4cfe-847c-44d413960998" containerID="8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad" exitCode=0
Sep 16 18:41:08.505327 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.504627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerDied","Data":"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"}
Sep 16 18:41:08.505327 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.505184    2576 scope.go:117] "RemoveContainer" containerID="8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"
Sep 16 18:41:08.506498 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.506457    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:08.506584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.506537    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:08.506584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:08.506578    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:09.481790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:09.481760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:09.513593 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:09.513553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerStarted","Data":"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"}
Sep 16 18:41:10.096669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:10.096640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:41:10.097488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:10.097465    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:41:10.097598 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:10.097520    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:41:11.096565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:11.096532    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:41:11.096958 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:11.096585    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:41:12.096955 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:12.096903    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:41:12.097335 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:12.096975    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:41:13.096604 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.096565    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:41:13.096779 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.096618    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:41:13.302616 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.302580    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:41:13.945954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.945902    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:41:13.954960 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.954918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:13.960175 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.960152    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e8510f-serving-cert\""
Sep 16 18:41:13.982084 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:13.982060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:41:14.005511 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.005474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.005636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.005523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.005636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.005621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfcp\" (UniqueName: \"kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.005766 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.005661    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.097413 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.097381    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:41:14.097607 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.097439    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:41:14.106883 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.106854    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.107014 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.106899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.107014 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.107000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfcp\" (UniqueName: \"kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.107142 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:41:14.107026    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-serving-cert: secret "maas-api-e2e-inf-b-e8510f-serving-cert" not found
Sep 16 18:41:14.107142 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.107043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.107142 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:41:14.107096    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls podName:10e4b8ba-36ae-4111-a302-be6f290f297e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:14.607075029 +0000 UTC m=+802.027561881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls") pod "maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" (UID: "10e4b8ba-36ae-4111-a302-be6f290f297e") : secret "maas-api-e2e-inf-b-e8510f-serving-cert" not found
Sep 16 18:41:14.107666 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.107640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.109653 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.109635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.149570 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.149533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfcp\" (UniqueName: \"kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.612832 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.612797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.616585 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.616556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:14.871196 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:14.871123    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:15.101638 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:15.101610    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:41:15.102708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:15.102685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:41:15.198092 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:15.197997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:41:15.573521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:15.573439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" event={"ID":"10e4b8ba-36ae-4111-a302-be6f290f297e","Type":"ContainerStarted","Data":"16f1595b0bbebbb632ee337e260730006356e2d826977442798aabcd520fc5f8"}
Sep 16 18:41:16.581503 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:16.581471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" event={"ID":"10e4b8ba-36ae-4111-a302-be6f290f297e","Type":"ContainerStarted","Data":"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"}
Sep 16 18:41:16.581959 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:16.581568    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:16.649867 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:16.649818    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" podStartSLOduration=3.261051504 podStartE2EDuration="3.649801569s" totalImagesPullingTime="388.750065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:16.63533505 +0000 UTC m=+804.055821920" watchObservedRunningTime="2026-09-16 18:41:16.649801569 +0000 UTC m=+804.070288428"
Sep 16 18:41:18.598708 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.598673    2576 generic.go:356] "Generic (PLEG): container finished" podID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerID="9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc" exitCode=0
Sep 16 18:41:18.599211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.598711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerDied","Data":"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"}
Sep 16 18:41:18.599279 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.599227    2576 scope.go:117] "RemoveContainer" containerID="9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"
Sep 16 18:41:18.600372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.600337    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:18.600446 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.600430    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:18.600488 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:18.600473    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 16 18:41:19.607300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:19.607262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerStarted","Data":"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"}
Sep 16 18:41:20.265090 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:20.265061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:41:20.266179 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:20.266148    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:20.266339 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:20.266211    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:20.357580 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:20.357547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:41:21.265784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:21.265753    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:21.266281 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:21.265808    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:22.265443 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:22.265410    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:22.265608 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:22.265467    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:22.592786 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:22.592713    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:41:23.266013 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:23.265980    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:23.266190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:23.266034    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:24.265389 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:24.265356    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:24.265775 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:24.265410    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:25.265674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:25.265638    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:25.266072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:25.265696    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:26.265573 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:26.265529    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:41:26.265759 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:26.265589    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:41:27.269301 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:27.269270    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:41:27.270316 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:27.270297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:41:27.879987 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:27.879914    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:27.880239 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:27.880215    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" podUID="c4d1648d-7972-4c1b-a949-61afc6139644" containerName="maas-api" containerID="cri-o://4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420" gracePeriod=30
Sep 16 18:41:28.144294 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.144269    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:28.253855 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.253818    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2m6l\" (UniqueName: \"kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l\") pod \"c4d1648d-7972-4c1b-a949-61afc6139644\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") "
Sep 16 18:41:28.253855 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.253873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") pod \"c4d1648d-7972-4c1b-a949-61afc6139644\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") "
Sep 16 18:41:28.254127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.253914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs\") pod \"c4d1648d-7972-4c1b-a949-61afc6139644\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") "
Sep 16 18:41:28.254127 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.253963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca\") pod \"c4d1648d-7972-4c1b-a949-61afc6139644\" (UID: \"c4d1648d-7972-4c1b-a949-61afc6139644\") "
Sep 16 18:41:28.254376 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.254341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4d1648d-7972-4c1b-a949-61afc6139644" (UID: "c4d1648d-7972-4c1b-a949-61afc6139644"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:28.254509 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.254491    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d1648d-7972-4c1b-a949-61afc6139644-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:41:28.256370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.256339    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c4d1648d-7972-4c1b-a949-61afc6139644" (UID: "c4d1648d-7972-4c1b-a949-61afc6139644"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:41:28.256657 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.256635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4d1648d-7972-4c1b-a949-61afc6139644" (UID: "c4d1648d-7972-4c1b-a949-61afc6139644"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:41:28.256713 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.256639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l" (OuterVolumeSpecName: "kube-api-access-k2m6l") pod "c4d1648d-7972-4c1b-a949-61afc6139644" (UID: "c4d1648d-7972-4c1b-a949-61afc6139644"). InnerVolumeSpecName "kube-api-access-k2m6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:28.355583 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.355559    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2m6l\" (UniqueName: \"kubernetes.io/projected/c4d1648d-7972-4c1b-a949-61afc6139644-kube-api-access-k2m6l\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:41:28.355583 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.355586    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:41:28.355918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.355595    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4d1648d-7972-4c1b-a949-61afc6139644-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:41:28.681820 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.681787    2576 generic.go:356] "Generic (PLEG): container finished" podID="c4d1648d-7972-4c1b-a949-61afc6139644" containerID="4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420" exitCode=0
Sep 16 18:41:28.682011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.681830    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"
Sep 16 18:41:28.682011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.681873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" event={"ID":"c4d1648d-7972-4c1b-a949-61afc6139644","Type":"ContainerDied","Data":"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"}
Sep 16 18:41:28.682011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.681941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8" event={"ID":"c4d1648d-7972-4c1b-a949-61afc6139644","Type":"ContainerDied","Data":"0153a2a08f55152104c736737e6c616ddf1224b376d50f8ee0598bc89e6a39c3"}
Sep 16 18:41:28.682011 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.681967    2576 scope.go:117] "RemoveContainer" containerID="4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"
Sep 16 18:41:28.692432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.692417    2576 scope.go:117] "RemoveContainer" containerID="4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"
Sep 16 18:41:28.692715 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:41:28.692695    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420\": container with ID starting with 4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420 not found: ID does not exist" containerID="4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"
Sep 16 18:41:28.692786 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.692727    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420"} err="failed to get container status \"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420\": rpc error: code = NotFound desc = could not find container \"4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420\": container with ID starting with 4a46489df3b91c20850898efcf96e1b9e0b4289143392b2fe849a86e73b80420 not found: ID does not exist"
Sep 16 18:41:28.736464 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.736440    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:28.743720 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:28.743691    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b7e002c7-6c8f5c5999-f6lq8"]
Sep 16 18:41:29.012477 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:29.012399    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d1648d-7972-4c1b-a949-61afc6139644" path="/var/lib/kubelet/pods/c4d1648d-7972-4c1b-a949-61afc6139644/volumes"
Sep 16 18:41:44.970678 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.970648    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:41:44.971424 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.971402    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d1648d-7972-4c1b-a949-61afc6139644" containerName="maas-api"
Sep 16 18:41:44.971521 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.971427    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d1648d-7972-4c1b-a949-61afc6139644" containerName="maas-api"
Sep 16 18:41:44.971575 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.971533    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d1648d-7972-4c1b-a949-61afc6139644" containerName="maas-api"
Sep 16 18:41:44.974671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.974648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:44.978464 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.978415    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b88c0110-serving-cert\""
Sep 16 18:41:44.996115 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:44.996093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:41:45.008219 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.008199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.008344 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.008225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c86k\" (UniqueName: \"kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.008344 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.008307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.008344 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.008333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.109168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.109141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.109168 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.109173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.109378 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.109358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.109432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.109395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c86k\" (UniqueName: \"kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.109990 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.109962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.111620 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.111597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.111706 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.111686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.139264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.139238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c86k\" (UniqueName: \"kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k\") pod \"maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.285151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.285089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:45.605756 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.605733    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:41:45.607273 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:41:45.607252    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720f6520_e984_4126_b261_5b3e12387a79.slice/crio-7ba5290b86374c0d24fd4338aeec04d4d69120195cebdbc138df4d58705ddce3 WatchSource:0}: Error finding container 7ba5290b86374c0d24fd4338aeec04d4d69120195cebdbc138df4d58705ddce3: Status 404 returned error can't find the container with id 7ba5290b86374c0d24fd4338aeec04d4d69120195cebdbc138df4d58705ddce3
Sep 16 18:41:45.785835 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:45.785802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" event={"ID":"720f6520-e984-4126-b261-5b3e12387a79","Type":"ContainerStarted","Data":"7ba5290b86374c0d24fd4338aeec04d4d69120195cebdbc138df4d58705ddce3"}
Sep 16 18:41:46.794534 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:46.794488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" event={"ID":"720f6520-e984-4126-b261-5b3e12387a79","Type":"ContainerStarted","Data":"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"}
Sep 16 18:41:46.794915 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:46.794581    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:41:46.839157 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:46.839107    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" podStartSLOduration=2.48506258 podStartE2EDuration="2.839093743s" totalImagesPullingTime="354.031163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:46.837156887 +0000 UTC m=+834.257643749" watchObservedRunningTime="2026-09-16 18:41:46.839093743 +0000 UTC m=+834.259580603"
Sep 16 18:41:52.806580 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:41:52.806552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:42:13.552894 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.552828    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:42:13.553526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.553067    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" podUID="720f6520-e984-4126-b261-5b3e12387a79" containerName="maas-api" containerID="cri-o://4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f" gracePeriod=30
Sep 16 18:42:13.795109 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.795086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:42:13.882750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.882670    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls\") pod \"720f6520-e984-4126-b261-5b3e12387a79\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") "
Sep 16 18:42:13.882750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.882743    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs\") pod \"720f6520-e984-4126-b261-5b3e12387a79\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") "
Sep 16 18:42:13.883062 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.882806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca\") pod \"720f6520-e984-4126-b261-5b3e12387a79\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") "
Sep 16 18:42:13.883062 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.882858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c86k\" (UniqueName: \"kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k\") pod \"720f6520-e984-4126-b261-5b3e12387a79\" (UID: \"720f6520-e984-4126-b261-5b3e12387a79\") "
Sep 16 18:42:13.883291 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.883263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "720f6520-e984-4126-b261-5b3e12387a79" (UID: "720f6520-e984-4126-b261-5b3e12387a79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:42:13.885110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.885085    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "720f6520-e984-4126-b261-5b3e12387a79" (UID: "720f6520-e984-4126-b261-5b3e12387a79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:42:13.885211 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.885154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "720f6520-e984-4126-b261-5b3e12387a79" (UID: "720f6520-e984-4126-b261-5b3e12387a79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:42:13.885309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.885288    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k" (OuterVolumeSpecName: "kube-api-access-5c86k") pod "720f6520-e984-4126-b261-5b3e12387a79" (UID: "720f6520-e984-4126-b261-5b3e12387a79"). InnerVolumeSpecName "kube-api-access-5c86k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:42:13.957824 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.957798    2576 generic.go:356] "Generic (PLEG): container finished" podID="720f6520-e984-4126-b261-5b3e12387a79" containerID="4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f" exitCode=0
Sep 16 18:42:13.957958 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.957853    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"
Sep 16 18:42:13.957958 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.957887    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" event={"ID":"720f6520-e984-4126-b261-5b3e12387a79","Type":"ContainerDied","Data":"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"}
Sep 16 18:42:13.958037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.957984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb" event={"ID":"720f6520-e984-4126-b261-5b3e12387a79","Type":"ContainerDied","Data":"7ba5290b86374c0d24fd4338aeec04d4d69120195cebdbc138df4d58705ddce3"}
Sep 16 18:42:13.958037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.958010    2576 scope.go:117] "RemoveContainer" containerID="4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"
Sep 16 18:42:13.972998 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.969303    2576 scope.go:117] "RemoveContainer" containerID="4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"
Sep 16 18:42:13.973210 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:13.973180    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f\": container with ID starting with 4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f not found: ID does not exist" containerID="4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"
Sep 16 18:42:13.973326 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.973220    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f"} err="failed to get container status \"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f\": rpc error: code = NotFound desc = could not find container \"4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f\": container with ID starting with 4809dd8ed194c9facb6427748e001426318eedacb322a50fc6a6c77b0c01560f not found: ID does not exist"
Sep 16 18:42:13.984574 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.984554    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/720f6520-e984-4126-b261-5b3e12387a79-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:13.984665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.984575    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c86k\" (UniqueName: \"kubernetes.io/projected/720f6520-e984-4126-b261-5b3e12387a79-kube-api-access-5c86k\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:13.984665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.984592    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:13.984665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.984604    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/720f6520-e984-4126-b261-5b3e12387a79-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:13.985389 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.985370    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:42:13.992892 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:13.992868    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b88c0110-6ddf59c75c-lpxlb"]
Sep 16 18:42:15.014585 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:15.014504    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="720f6520-e984-4126-b261-5b3e12387a79" path="/var/lib/kubelet/pods/720f6520-e984-4126-b261-5b3e12387a79/volumes"
Sep 16 18:42:24.542084 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.542052    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:24.542734 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.542714    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720f6520-e984-4126-b261-5b3e12387a79" containerName="maas-api"
Sep 16 18:42:24.542832 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.542736    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="720f6520-e984-4126-b261-5b3e12387a79" containerName="maas-api"
Sep 16 18:42:24.542888 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.542876    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="720f6520-e984-4126-b261-5b3e12387a79" containerName="maas-api"
Sep 16 18:42:24.544837 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.544818    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.549207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.549007    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c6d0650c-serving-cert\""
Sep 16 18:42:24.559962 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.559918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:24.578594 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.578559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw6sk\" (UniqueName: \"kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.578695 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.578655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.578738 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.578706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.578786 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.578775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.679378 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.679340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6sk\" (UniqueName: \"kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.679535 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.679401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.679535 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.679444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.679535 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.679505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.679638 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:24.679591    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-serving-cert: secret "maas-api-e2e-ait-c6d0650c-serving-cert" not found
Sep 16 18:42:24.679686 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:24.679675    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls podName:e5cb6479-c27d-4abc-aa28-df1bab1eab75 nodeName:}" failed. No retries permitted until 2026-09-16 18:42:25.179651725 +0000 UTC m=+872.600138562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls") pod "maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75") : secret "maas-api-e2e-ait-c6d0650c-serving-cert" not found
Sep 16 18:42:24.680215 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.680154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.682039 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.682016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:24.690627 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:24.690608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw6sk\" (UniqueName: \"kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:25.184224 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:25.184190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:25.184387 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:25.184320    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-serving-cert: secret "maas-api-e2e-ait-c6d0650c-serving-cert" not found
Sep 16 18:42:25.184431 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:25.184388    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls podName:e5cb6479-c27d-4abc-aa28-df1bab1eab75 nodeName:}" failed. No retries permitted until 2026-09-16 18:42:26.184372001 +0000 UTC m=+873.604858838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls") pod "maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75") : secret "maas-api-e2e-ait-c6d0650c-serving-cert" not found
Sep 16 18:42:26.190871 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:26.190823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:26.193404 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:26.193385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"maas-api-e2e-ait-c6d0650c-57969684db-cr8qj\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:26.358554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:26.358526    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:26.643911 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:26.643884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:26.645024 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:42:26.644996    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5cb6479_c27d_4abc_aa28_df1bab1eab75.slice/crio-819ed9168d6837e266836403617f8ae8c4ee3f86d5d57ad12f20045f87458d99 WatchSource:0}: Error finding container 819ed9168d6837e266836403617f8ae8c4ee3f86d5d57ad12f20045f87458d99: Status 404 returned error can't find the container with id 819ed9168d6837e266836403617f8ae8c4ee3f86d5d57ad12f20045f87458d99
Sep 16 18:42:27.037698 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:27.037660    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" event={"ID":"e5cb6479-c27d-4abc-aa28-df1bab1eab75","Type":"ContainerStarted","Data":"819ed9168d6837e266836403617f8ae8c4ee3f86d5d57ad12f20045f87458d99"}
Sep 16 18:42:28.049037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:28.049004    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" event={"ID":"e5cb6479-c27d-4abc-aa28-df1bab1eab75","Type":"ContainerStarted","Data":"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"}
Sep 16 18:42:28.049415 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:28.049067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:28.076022 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:28.075973    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" podStartSLOduration=3.682047451 podStartE2EDuration="4.075958261s" totalImagesPullingTime="393.91081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:28.074958514 +0000 UTC m=+875.495445370" watchObservedRunningTime="2026-09-16 18:42:28.075958261 +0000 UTC m=+875.496445121"
Sep 16 18:42:34.061138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:34.061111    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:52.898372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:52.898340    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:52.899293 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:52.899241    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" podUID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" containerName="maas-api" containerID="cri-o://d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f" gracePeriod=30
Sep 16 18:42:53.171209 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.170486    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:53.213974 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.213914    2576 generic.go:356] "Generic (PLEG): container finished" podID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" containerID="d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f" exitCode=0
Sep 16 18:42:53.214148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.213988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" event={"ID":"e5cb6479-c27d-4abc-aa28-df1bab1eab75","Type":"ContainerDied","Data":"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"}
Sep 16 18:42:53.214148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.214057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj" event={"ID":"e5cb6479-c27d-4abc-aa28-df1bab1eab75","Type":"ContainerDied","Data":"819ed9168d6837e266836403617f8ae8c4ee3f86d5d57ad12f20045f87458d99"}
Sep 16 18:42:53.214148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.213992    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"
Sep 16 18:42:53.214148 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.214094    2576 scope.go:117] "RemoveContainer" containerID="d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"
Sep 16 18:42:53.226116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.226100    2576 scope.go:117] "RemoveContainer" containerID="d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"
Sep 16 18:42:53.226422 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:42:53.226384    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f\": container with ID starting with d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f not found: ID does not exist" containerID="d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"
Sep 16 18:42:53.226538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.226426    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f"} err="failed to get container status \"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f\": rpc error: code = NotFound desc = could not find container \"d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f\": container with ID starting with d2ff46690b7f663b318951d9cde677bfa2a3091d454c78b6bc750fe50784269f not found: ID does not exist"
Sep 16 18:42:53.241266 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.241245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw6sk\" (UniqueName: \"kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk\") pod \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") "
Sep 16 18:42:53.241394 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.241276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") pod \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") "
Sep 16 18:42:53.241394 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.241310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs\") pod \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") "
Sep 16 18:42:53.241394 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.241347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca\") pod \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\" (UID: \"e5cb6479-c27d-4abc-aa28-df1bab1eab75\") "
Sep 16 18:42:53.241755 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.241725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5cb6479-c27d-4abc-aa28-df1bab1eab75" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:42:53.243589 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.243562    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5cb6479-c27d-4abc-aa28-df1bab1eab75" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:42:53.243589 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.243568    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk" (OuterVolumeSpecName: "kube-api-access-gw6sk") pod "e5cb6479-c27d-4abc-aa28-df1bab1eab75" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75"). InnerVolumeSpecName "kube-api-access-gw6sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:42:53.243719 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.243572    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e5cb6479-c27d-4abc-aa28-df1bab1eab75" (UID: "e5cb6479-c27d-4abc-aa28-df1bab1eab75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:42:53.343420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.343386    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw6sk\" (UniqueName: \"kubernetes.io/projected/e5cb6479-c27d-4abc-aa28-df1bab1eab75-kube-api-access-gw6sk\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:53.343420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.343418    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:53.343420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.343431    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5cb6479-c27d-4abc-aa28-df1bab1eab75-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:53.343420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.343443    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cb6479-c27d-4abc-aa28-df1bab1eab75-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:42:53.544752 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.544722    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:53.547619 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:53.547597    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6d0650c-57969684db-cr8qj"]
Sep 16 18:42:55.011494 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:42:55.011464    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" path="/var/lib/kubelet/pods/e5cb6479-c27d-4abc-aa28-df1bab1eab75/volumes"
Sep 16 18:43:03.390536 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.390497    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:03.391177 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.390985    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" containerName="maas-api"
Sep 16 18:43:03.391177 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.390998    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" containerName="maas-api"
Sep 16 18:43:03.391177 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.391067    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cb6479-c27d-4abc-aa28-df1bab1eab75" containerName="maas-api"
Sep 16 18:43:03.395533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.395513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.398495 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.398470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0c558678-serving-cert\""
Sep 16 18:43:03.409363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.409341    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:03.443662 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.443631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.443789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.443678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.443789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.443703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.443789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.443761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mj2x\" (UniqueName: \"kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.544465 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.544433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.544632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.544481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.544632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.544500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.544632 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:43:03.544602    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-serving-cert: secret "maas-api-e2e-derive-0c558678-serving-cert" not found
Sep 16 18:43:03.544805 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.544627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mj2x\" (UniqueName: \"kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.544805 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:43:03.544681    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls podName:9558531d-fc53-446d-9326-e08f6e4eeda3 nodeName:}" failed. No retries permitted until 2026-09-16 18:43:04.044658872 +0000 UTC m=+911.465145710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls") pod "maas-api-e2e-derive-0c558678-b9d688686-tjsxw" (UID: "9558531d-fc53-446d-9326-e08f6e4eeda3") : secret "maas-api-e2e-derive-0c558678-serving-cert" not found
Sep 16 18:43:03.545249 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.545227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.547391 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.547367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:03.560317 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:03.560294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mj2x\" (UniqueName: \"kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:04.049013 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:04.048977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:04.051552 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:04.051523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") pod \"maas-api-e2e-derive-0c558678-b9d688686-tjsxw\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:04.306761 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:04.306688    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:04.596760 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:04.596730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:05.295104 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:05.295068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" event={"ID":"9558531d-fc53-446d-9326-e08f6e4eeda3","Type":"ContainerStarted","Data":"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"}
Sep 16 18:43:05.295273 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:05.295129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" event={"ID":"9558531d-fc53-446d-9326-e08f6e4eeda3","Type":"ContainerStarted","Data":"d317469027575c50ff70f88e2d8087b5d76bd954f8e14a5b4346c6bfb00a053f"}
Sep 16 18:43:05.295273 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:05.295155    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:05.317076 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:05.317021    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" podStartSLOduration=1.808662414 podStartE2EDuration="2.317007172s" totalImagesPullingTime="508.344758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:05.315111135 +0000 UTC m=+912.735597995" watchObservedRunningTime="2026-09-16 18:43:05.317007172 +0000 UTC m=+912.737494032"
Sep 16 18:43:11.309016 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:11.308979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:28.152545 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.152502    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:28.153037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.152781    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" podUID="9558531d-fc53-446d-9326-e08f6e4eeda3" containerName="maas-api" containerID="cri-o://0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c" gracePeriod=30
Sep 16 18:43:28.419546 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.419516    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:28.441276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.441241    2576 generic.go:356] "Generic (PLEG): container finished" podID="9558531d-fc53-446d-9326-e08f6e4eeda3" containerID="0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c" exitCode=0
Sep 16 18:43:28.441470 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.441298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" event={"ID":"9558531d-fc53-446d-9326-e08f6e4eeda3","Type":"ContainerDied","Data":"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"}
Sep 16 18:43:28.441470 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.441351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw" event={"ID":"9558531d-fc53-446d-9326-e08f6e4eeda3","Type":"ContainerDied","Data":"d317469027575c50ff70f88e2d8087b5d76bd954f8e14a5b4346c6bfb00a053f"}
Sep 16 18:43:28.441470 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.441387    2576 scope.go:117] "RemoveContainer" containerID="0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"
Sep 16 18:43:28.442142 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.441573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"
Sep 16 18:43:28.463185 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.463162    2576 scope.go:117] "RemoveContainer" containerID="0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"
Sep 16 18:43:28.463534 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:43:28.463508    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c\": container with ID starting with 0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c not found: ID does not exist" containerID="0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"
Sep 16 18:43:28.463587 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.463549    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c"} err="failed to get container status \"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c\": rpc error: code = NotFound desc = could not find container \"0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c\": container with ID starting with 0b95c3c6bcd8abe418e2863e877d7a40c5f04d4a752265f8c9564d884bc4660c not found: ID does not exist"
Sep 16 18:43:28.584853 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.584819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") pod \"9558531d-fc53-446d-9326-e08f6e4eeda3\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") "
Sep 16 18:43:28.585021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.584861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mj2x\" (UniqueName: \"kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x\") pod \"9558531d-fc53-446d-9326-e08f6e4eeda3\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") "
Sep 16 18:43:28.585021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.584953    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs\") pod \"9558531d-fc53-446d-9326-e08f6e4eeda3\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") "
Sep 16 18:43:28.585151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.585031    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca\") pod \"9558531d-fc53-446d-9326-e08f6e4eeda3\" (UID: \"9558531d-fc53-446d-9326-e08f6e4eeda3\") "
Sep 16 18:43:28.585692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.585598    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9558531d-fc53-446d-9326-e08f6e4eeda3" (UID: "9558531d-fc53-446d-9326-e08f6e4eeda3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:28.587369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.587337    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9558531d-fc53-446d-9326-e08f6e4eeda3" (UID: "9558531d-fc53-446d-9326-e08f6e4eeda3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:28.587530 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.587509    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9558531d-fc53-446d-9326-e08f6e4eeda3" (UID: "9558531d-fc53-446d-9326-e08f6e4eeda3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:28.587675 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.587656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x" (OuterVolumeSpecName: "kube-api-access-6mj2x") pod "9558531d-fc53-446d-9326-e08f6e4eeda3" (UID: "9558531d-fc53-446d-9326-e08f6e4eeda3"). InnerVolumeSpecName "kube-api-access-6mj2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:28.686352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.686280    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:28.686352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.686306    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mj2x\" (UniqueName: \"kubernetes.io/projected/9558531d-fc53-446d-9326-e08f6e4eeda3-kube-api-access-6mj2x\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:28.686352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.686316    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9558531d-fc53-446d-9326-e08f6e4eeda3-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:28.686352 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.686324    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9558531d-fc53-446d-9326-e08f6e4eeda3-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:28.772382 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.772354    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:28.781574 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:28.781551    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0c558678-b9d688686-tjsxw"]
Sep 16 18:43:29.011403 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:29.011318    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9558531d-fc53-446d-9326-e08f6e4eeda3" path="/var/lib/kubelet/pods/9558531d-fc53-446d-9326-e08f6e4eeda3/volumes"
Sep 16 18:43:45.818202 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:45.818161    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:43:45.818752 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:45.818444    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" podUID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" containerName="maas-api" containerID="cri-o://fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530" gracePeriod=30
Sep 16 18:43:46.086191 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.086103    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:43:46.148302 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.148276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8js65\" (UniqueName: \"kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65\") pod \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") "
Sep 16 18:43:46.148454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.148391    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") pod \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") "
Sep 16 18:43:46.148454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.148424    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs\") pod \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") "
Sep 16 18:43:46.148454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.148445    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca\") pod \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\" (UID: \"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741\") "
Sep 16 18:43:46.148886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.148858    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:46.151193 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.151156    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:46.151336 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.151208    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65" (OuterVolumeSpecName: "kube-api-access-8js65") pod "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741"). InnerVolumeSpecName "kube-api-access-8js65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:46.151336 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.151278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" (UID: "cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:46.251116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.251052    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.251116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.251090    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.251116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.251104    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.251116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.251123    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8js65\" (UniqueName: \"kubernetes.io/projected/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741-kube-api-access-8js65\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.550906 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.550868    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" containerID="fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530" exitCode=0
Sep 16 18:43:46.551115 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.550917    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"
Sep 16 18:43:46.551115 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.550946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" event={"ID":"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741","Type":"ContainerDied","Data":"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"}
Sep 16 18:43:46.551115 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.551000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49" event={"ID":"cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741","Type":"ContainerDied","Data":"dc6940bee53cf100d3414890c0266007821176793f846958733837ef86783d2c"}
Sep 16 18:43:46.551115 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.551027    2576 scope.go:117] "RemoveContainer" containerID="fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"
Sep 16 18:43:46.561747 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.561730    2576 scope.go:117] "RemoveContainer" containerID="fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"
Sep 16 18:43:46.562037 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:43:46.562015    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530\": container with ID starting with fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530 not found: ID does not exist" containerID="fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"
Sep 16 18:43:46.562101 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.562045    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530"} err="failed to get container status \"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530\": rpc error: code = NotFound desc = could not find container \"fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530\": container with ID starting with fc6ef50d87972970de6f2b3507a55ef763debb55268b472b816bb9387c180530 not found: ID does not exist"
Sep 16 18:43:46.585920 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.585900    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:43:46.589959 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:46.589940    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-90ea7d-79fb9f64cb-clt49"]
Sep 16 18:43:47.012324 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:43:47.012298    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" path="/var/lib/kubelet/pods/cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741/volumes"
Sep 16 18:44:39.275398 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:39.275350    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:44:39.275913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:39.275609    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" containerID="cri-o://1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6" gracePeriod=30
Sep 16 18:44:42.272124 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:42.272083    2576 patch_prober.go:28] interesting pod/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:44:42.272493 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:42.272190    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:44:44.541200 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.541134    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:44:44.701101 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701114    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701158    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkb82\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701289    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701325    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701406    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701455    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:44.701544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701514    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data\") pod \"49b31970-25a6-4961-82fb-f058ac2eaf58\" (UID: \"49b31970-25a6-4961-82fb-f058ac2eaf58\") "
Sep 16 18:44:44.701878 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.701827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data" (OuterVolumeSpecName: "istio-data") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:44.702032 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.702012    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49b31970-25a6-4961-82fb-f058ac2eaf58-istiod-ca-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.702086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.702039    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.702149 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.702094    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-data\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.702242 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.702217    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-workload-socket\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.702310 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.702243    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-credential-socket\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.703692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.703660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:44:44.703807 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.703686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82" (OuterVolumeSpecName: "kube-api-access-kkb82") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "kube-api-access-kkb82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:44:44.703807 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.703693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token" (OuterVolumeSpecName: "istio-token") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:44:44.703972 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.703950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49b31970-25a6-4961-82fb-f058ac2eaf58" (UID: "49b31970-25a6-4961-82fb-f058ac2eaf58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:44.803264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.803200    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-podinfo\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.803264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.803222    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-envoy\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.803264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.803231    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-istio-token\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.803264 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.803239    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkb82\" (UniqueName: \"kubernetes.io/projected/49b31970-25a6-4961-82fb-f058ac2eaf58-kube-api-access-kkb82\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.899658 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.899628    2576 generic.go:356] "Generic (PLEG): container finished" podID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerID="1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6" exitCode=0
Sep 16 18:44:44.899901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.899684    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"
Sep 16 18:44:44.899901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.899721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerDied","Data":"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"}
Sep 16 18:44:44.899901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.899800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4" event={"ID":"49b31970-25a6-4961-82fb-f058ac2eaf58","Type":"ContainerDied","Data":"c6266ae6f96cc69385c804e4f5bbffce51102b9e8927825ac99953b1b20ccdfd"}
Sep 16 18:44:44.899901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.899825    2576 scope.go:117] "RemoveContainer" containerID="1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"
Sep 16 18:44:44.910099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.910024    2576 scope.go:117] "RemoveContainer" containerID="9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"
Sep 16 18:44:44.919202 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.919186    2576 scope.go:117] "RemoveContainer" containerID="1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"
Sep 16 18:44:44.919491 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:44:44.919470    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6\": container with ID starting with 1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6 not found: ID does not exist" containerID="1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"
Sep 16 18:44:44.919565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.919498    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6"} err="failed to get container status \"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6\": rpc error: code = NotFound desc = could not find container \"1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6\": container with ID starting with 1d71dbb991ff4adf1ccf1bc242019cf1a15b6a3ffe5642d89bafd2bfba3345e6 not found: ID does not exist"
Sep 16 18:44:44.919565 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.919514    2576 scope.go:117] "RemoveContainer" containerID="9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"
Sep 16 18:44:44.919781 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:44:44.919762    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc\": container with ID starting with 9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc not found: ID does not exist" containerID="9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"
Sep 16 18:44:44.919848 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.919786    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc"} err="failed to get container status \"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc\": rpc error: code = NotFound desc = could not find container \"9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc\": container with ID starting with 9ef5a6a9a791e8c320e1eb0bfead46cdb00f989b59149c19495cb6c9ae3768dc not found: ID does not exist"
Sep 16 18:44:44.924507 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.924484    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:44:44.931103 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:44.931083    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-90ea7d-openshift-default-6f8c8bbf7d-v2vr4"]
Sep 16 18:44:45.011810 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:45.011782    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" path="/var/lib/kubelet/pods/49b31970-25a6-4961-82fb-f058ac2eaf58/volumes"
Sep 16 18:44:46.619448 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619400    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:44:46.619915 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619899    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" containerName="maas-api"
Sep 16 18:44:46.619984 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619917    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" containerName="maas-api"
Sep 16 18:44:46.619984 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619958    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.619984 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619964    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.619984 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619975    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9558531d-fc53-446d-9326-e08f6e4eeda3" containerName="maas-api"
Sep 16 18:44:46.619984 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619980    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9558531d-fc53-446d-9326-e08f6e4eeda3" containerName="maas-api"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619991    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.619997    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.620087    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.620101    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd6ec2c-a67f-4b40-89b8-3d32e3c6a741" containerName="maas-api"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.620108    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b31970-25a6-4961-82fb-f058ac2eaf58" containerName="istio-proxy"
Sep 16 18:44:46.620138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.620114    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9558531d-fc53-446d-9326-e08f6e4eeda3" containerName="maas-api"
Sep 16 18:44:46.624981 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.624958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.627188 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.627163    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-16ae98-serving-cert\""
Sep 16 18:44:46.633037 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.633014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:44:46.720794 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.720745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.721154 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.721130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bczn2\" (UniqueName: \"kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.721327 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.721310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.721549 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.721526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.822375 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.822345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.822545 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.822396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bczn2\" (UniqueName: \"kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.822545 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.822418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.822545 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.822463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.822978 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.822957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.825061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.825045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.825197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.825179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.831358 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.831335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bczn2\" (UniqueName: \"kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2\") pod \"maas-api-e2e-explicit-16ae98-59d757947-wd448\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:46.939323 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:46.939297    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:47.224105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:47.224055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:44:47.227786 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:44:47.227746    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64792d94_d250_4394_8fa7_b6e0e9ce0819.slice/crio-ae786373bf937c0d878d00f3182749c990b5ebcd8172af137d7004c621c3e3e5 WatchSource:0}: Error finding container ae786373bf937c0d878d00f3182749c990b5ebcd8172af137d7004c621c3e3e5: Status 404 returned error can't find the container with id ae786373bf937c0d878d00f3182749c990b5ebcd8172af137d7004c621c3e3e5
Sep 16 18:44:47.920448 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:47.920407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" event={"ID":"64792d94-d250-4394-8fa7-b6e0e9ce0819","Type":"ContainerStarted","Data":"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"}
Sep 16 18:44:47.920879 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:47.920466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" event={"ID":"64792d94-d250-4394-8fa7-b6e0e9ce0819","Type":"ContainerStarted","Data":"ae786373bf937c0d878d00f3182749c990b5ebcd8172af137d7004c621c3e3e5"}
Sep 16 18:44:47.920879 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:47.920486    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:44:47.941380 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:47.941328    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" podStartSLOduration=1.5765042500000002 podStartE2EDuration="1.941314323s" totalImagesPullingTime="364.810073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:47.938738593 +0000 UTC m=+1015.359225476" watchObservedRunningTime="2026-09-16 18:44:47.941314323 +0000 UTC m=+1015.361801182"
Sep 16 18:44:53.933762 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:44:53.933731    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:45:29.280340 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.280300    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:45:29.285550 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.285527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.288164 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.288143    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6498ead1-serving-cert\""
Sep 16 18:45:29.297852 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.297825    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:45:29.427872 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.427846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.428072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.427897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vrw\" (UniqueName: \"kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.428072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.427969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.428150 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.428078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.528784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.528755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.528969 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.528804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.528969 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.528838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vrw\" (UniqueName: \"kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.528969 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.528857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.528969 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:45:29.528907    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-serving-cert: secret "maas-api-e2e-mt-6498ead1-serving-cert" not found
Sep 16 18:45:29.529140 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:45:29.529006    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls podName:c3f50fba-b2c2-4859-95fa-665e0f9b3c38 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:30.028989527 +0000 UTC m=+1057.449476369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls") pod "maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" (UID: "c3f50fba-b2c2-4859-95fa-665e0f9b3c38") : secret "maas-api-e2e-mt-6498ead1-serving-cert" not found
Sep 16 18:45:29.529549 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.529531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.531518 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.531451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:29.537903 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:29.537881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vrw\" (UniqueName: \"kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:30.034977 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:30.034925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:30.037432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:30.037402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") pod \"maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:30.198739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:30.198705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:30.476538 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:30.476508    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:45:30.478352 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:45:30.478331    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f50fba_b2c2_4859_95fa_665e0f9b3c38.slice/crio-40f09504751645087168ba912670fd3ebcd88da21ef6b274aa2e1097761cc42d WatchSource:0}: Error finding container 40f09504751645087168ba912670fd3ebcd88da21ef6b274aa2e1097761cc42d: Status 404 returned error can't find the container with id 40f09504751645087168ba912670fd3ebcd88da21ef6b274aa2e1097761cc42d
Sep 16 18:45:31.189195 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:31.189163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" event={"ID":"c3f50fba-b2c2-4859-95fa-665e0f9b3c38","Type":"ContainerStarted","Data":"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"}
Sep 16 18:45:31.189359 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:31.189218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" event={"ID":"c3f50fba-b2c2-4859-95fa-665e0f9b3c38","Type":"ContainerStarted","Data":"40f09504751645087168ba912670fd3ebcd88da21ef6b274aa2e1097761cc42d"}
Sep 16 18:45:31.189359 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:31.189264    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:45:31.213943 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:31.213879    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" podStartSLOduration=1.7163494639999999 podStartE2EDuration="2.213864802s" totalImagesPullingTime="497.515338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:31.211444092 +0000 UTC m=+1058.631930952" watchObservedRunningTime="2026-09-16 18:45:31.213864802 +0000 UTC m=+1058.634351712"
Sep 16 18:45:37.200164 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:45:37.200130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:46:08.722442 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:08.722389    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:46:08.723437 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:08.723384    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" podUID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" containerName="maas-api" containerID="cri-o://683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374" gracePeriod=30
Sep 16 18:46:08.975818 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:08.975760    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:46:09.129803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.129774    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7vrw\" (UniqueName: \"kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw\") pod \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") "
Sep 16 18:46:09.129979 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.129823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca\") pod \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") "
Sep 16 18:46:09.129979 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.129868    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") pod \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") "
Sep 16 18:46:09.129979 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.129922    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs\") pod \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\" (UID: \"c3f50fba-b2c2-4859-95fa-665e0f9b3c38\") "
Sep 16 18:46:09.130314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.130279    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3f50fba-b2c2-4859-95fa-665e0f9b3c38" (UID: "c3f50fba-b2c2-4859-95fa-665e0f9b3c38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:46:09.132171 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.132144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c3f50fba-b2c2-4859-95fa-665e0f9b3c38" (UID: "c3f50fba-b2c2-4859-95fa-665e0f9b3c38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:46:09.132301 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.132268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3f50fba-b2c2-4859-95fa-665e0f9b3c38" (UID: "c3f50fba-b2c2-4859-95fa-665e0f9b3c38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:46:09.132367 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.132335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw" (OuterVolumeSpecName: "kube-api-access-n7vrw") pod "c3f50fba-b2c2-4859-95fa-665e0f9b3c38" (UID: "c3f50fba-b2c2-4859-95fa-665e0f9b3c38"). InnerVolumeSpecName "kube-api-access-n7vrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:09.231309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.231220    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.231309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.231252    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.231309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.231262    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.231309 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.231270    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7vrw\" (UniqueName: \"kubernetes.io/projected/c3f50fba-b2c2-4859-95fa-665e0f9b3c38-kube-api-access-n7vrw\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.412947 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.412898    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" containerID="683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374" exitCode=0
Sep 16 18:46:09.413106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.412961    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"
Sep 16 18:46:09.413106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.412969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" event={"ID":"c3f50fba-b2c2-4859-95fa-665e0f9b3c38","Type":"ContainerDied","Data":"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"}
Sep 16 18:46:09.413106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.413031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc" event={"ID":"c3f50fba-b2c2-4859-95fa-665e0f9b3c38","Type":"ContainerDied","Data":"40f09504751645087168ba912670fd3ebcd88da21ef6b274aa2e1097761cc42d"}
Sep 16 18:46:09.413106 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.413054    2576 scope.go:117] "RemoveContainer" containerID="683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"
Sep 16 18:46:09.423798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.423784    2576 scope.go:117] "RemoveContainer" containerID="683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"
Sep 16 18:46:09.424051 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:46:09.424033    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374\": container with ID starting with 683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374 not found: ID does not exist" containerID="683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"
Sep 16 18:46:09.424111 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.424060    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374"} err="failed to get container status \"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374\": rpc error: code = NotFound desc = could not find container \"683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374\": container with ID starting with 683040e7a46ecf19a9810f7207a57c655846f800fa7119e9b176d4d0f6631374 not found: ID does not exist"
Sep 16 18:46:09.439851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.439829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:46:09.443782 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:09.443763    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6498ead1-7685dfc864-kn9vc"]
Sep 16 18:46:11.012361 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:11.012330    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" path="/var/lib/kubelet/pods/c3f50fba-b2c2-4859-95fa-665e0f9b3c38/volumes"
Sep 16 18:46:39.617763 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.617732    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:46:39.618214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.618186    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" containerName="maas-api"
Sep 16 18:46:39.618214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.618198    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" containerName="maas-api"
Sep 16 18:46:39.618294 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.618283    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f50fba-b2c2-4859-95fa-665e0f9b3c38" containerName="maas-api"
Sep 16 18:46:39.621061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.621037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.624225 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.624203    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-ee98a5-serving-cert\""
Sep 16 18:46:39.648725 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.648695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:46:39.717750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.717704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.718080 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.718056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.718322 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.718288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbzb\" (UniqueName: \"kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.718493 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.718403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.819917 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.819887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.820123 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.819972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.820123 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.820017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.820123 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:46:39.820056    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-serving-cert: secret "maas-api-e2e-shared-a-w2-ee98a5-serving-cert" not found
Sep 16 18:46:39.820123 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.820053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbzb\" (UniqueName: \"kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.820123 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:46:39.820126    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls podName:54a3607c-3f68-4407-87a7-82ab8206d7b4 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:40.320109705 +0000 UTC m=+1127.740596547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls") pod "maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" (UID: "54a3607c-3f68-4407-87a7-82ab8206d7b4") : secret "maas-api-e2e-shared-a-w2-ee98a5-serving-cert" not found
Sep 16 18:46:39.820699 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.820671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.822703 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.822681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:39.849491 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:39.849464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbzb\" (UniqueName: \"kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:40.326116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:40.326083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:40.328723 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:40.328700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:40.531247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:40.531206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:40.815262 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:40.815228    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:46:40.816533 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:46:40.816503    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a3607c_3f68_4407_87a7_82ab8206d7b4.slice/crio-b9eccc098def2f39c08ca64acc3641069efd5b5d0ce0f0dbfe4f7a643d799892 WatchSource:0}: Error finding container b9eccc098def2f39c08ca64acc3641069efd5b5d0ce0f0dbfe4f7a643d799892: Status 404 returned error can't find the container with id b9eccc098def2f39c08ca64acc3641069efd5b5d0ce0f0dbfe4f7a643d799892
Sep 16 18:46:40.818306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:40.818234    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:46:41.618632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:41.618541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" event={"ID":"54a3607c-3f68-4407-87a7-82ab8206d7b4","Type":"ContainerStarted","Data":"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"}
Sep 16 18:46:41.618632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:41.618601    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" event={"ID":"54a3607c-3f68-4407-87a7-82ab8206d7b4","Type":"ContainerStarted","Data":"b9eccc098def2f39c08ca64acc3641069efd5b5d0ce0f0dbfe4f7a643d799892"}
Sep 16 18:46:41.618632 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:41.618625    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:46:41.641480 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:41.641433    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" podStartSLOduration=2.27950751 podStartE2EDuration="2.641417188s" totalImagesPullingTime="361.909678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:41.638606367 +0000 UTC m=+1129.059093228" watchObservedRunningTime="2026-09-16 18:46:41.641417188 +0000 UTC m=+1129.061904047"
Sep 16 18:46:47.630756 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:46:47.630725    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:47:05.082329 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.082301    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:47:05.087305 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.087279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.089755 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.089726    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-8737d2-serving-cert\""
Sep 16 18:47:05.101146 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.101124    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:47:05.155285 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.155254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.155402 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.155289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.155402 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.155324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9rm\" (UniqueName: \"kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.155480 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.155421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.256893 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.256863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9rm\" (UniqueName: \"kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.257054 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.256967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.257054 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.257022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.257054 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.257045    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.257766 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.257743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.259618 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.259594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.259721 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.259596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.274200 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.274176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9rm\" (UniqueName: \"kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm\") pod \"maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.398480 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.398411    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:05.676458 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.676433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:47:05.677773 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:47:05.677746    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68637031_da63_464b_8de2_d455cf5f3d9e.slice/crio-538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c WatchSource:0}: Error finding container 538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c: Status 404 returned error can't find the container with id 538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c
Sep 16 18:47:05.767334 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:05.767298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" event={"ID":"68637031-da63-464b-8de2-d455cf5f3d9e","Type":"ContainerStarted","Data":"538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c"}
Sep 16 18:47:06.774891 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:06.774855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" event={"ID":"68637031-da63-464b-8de2-d455cf5f3d9e","Type":"ContainerStarted","Data":"0b07daf96824c55dccbbdd20c6d64f4dc8007e0ea0e377b0a14b76b46088b23b"}
Sep 16 18:47:06.775322 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:06.774909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:06.800634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:06.800578    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" podStartSLOduration=1.413374422 podStartE2EDuration="1.800563636s" totalImagesPullingTime="387.189214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:06.796922062 +0000 UTC m=+1154.217408945" watchObservedRunningTime="2026-09-16 18:47:06.800563636 +0000 UTC m=+1154.221050499"
Sep 16 18:47:08.916287 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:08.916253    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:47:08.916655 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:08.916488    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerName="maas-api" containerID="cri-o://149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90" gracePeriod=30
Sep 16 18:47:08.928601 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:08.928573    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.59:8443/health\": dial tcp 10.133.0.59:8443: connect: connection refused"
Sep 16 18:47:09.182898 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.182872    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:47:09.298732 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.298699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls\") pod \"64792d94-d250-4394-8fa7-b6e0e9ce0819\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") "
Sep 16 18:47:09.298901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.298811    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca\") pod \"64792d94-d250-4394-8fa7-b6e0e9ce0819\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") "
Sep 16 18:47:09.298901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.298841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bczn2\" (UniqueName: \"kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2\") pod \"64792d94-d250-4394-8fa7-b6e0e9ce0819\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") "
Sep 16 18:47:09.298901 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.298855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs\") pod \"64792d94-d250-4394-8fa7-b6e0e9ce0819\" (UID: \"64792d94-d250-4394-8fa7-b6e0e9ce0819\") "
Sep 16 18:47:09.299247 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.299219    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64792d94-d250-4394-8fa7-b6e0e9ce0819" (UID: "64792d94-d250-4394-8fa7-b6e0e9ce0819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:09.301062 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.301038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "64792d94-d250-4394-8fa7-b6e0e9ce0819" (UID: "64792d94-d250-4394-8fa7-b6e0e9ce0819"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:09.301149 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.301062    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64792d94-d250-4394-8fa7-b6e0e9ce0819" (UID: "64792d94-d250-4394-8fa7-b6e0e9ce0819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:09.301252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.301233    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2" (OuterVolumeSpecName: "kube-api-access-bczn2") pod "64792d94-d250-4394-8fa7-b6e0e9ce0819" (UID: "64792d94-d250-4394-8fa7-b6e0e9ce0819"). InnerVolumeSpecName "kube-api-access-bczn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:09.400395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.400363    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:09.400395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.400389    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64792d94-d250-4394-8fa7-b6e0e9ce0819-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:09.400395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.400399    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bczn2\" (UniqueName: \"kubernetes.io/projected/64792d94-d250-4394-8fa7-b6e0e9ce0819-kube-api-access-bczn2\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:09.400627 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.400407    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64792d94-d250-4394-8fa7-b6e0e9ce0819-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:09.795343 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.795307    2576 generic.go:356] "Generic (PLEG): container finished" podID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerID="149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90" exitCode=0
Sep 16 18:47:09.795559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.795360    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"
Sep 16 18:47:09.795559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.795383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" event={"ID":"64792d94-d250-4394-8fa7-b6e0e9ce0819","Type":"ContainerDied","Data":"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"}
Sep 16 18:47:09.795559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.795453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448" event={"ID":"64792d94-d250-4394-8fa7-b6e0e9ce0819","Type":"ContainerDied","Data":"ae786373bf937c0d878d00f3182749c990b5ebcd8172af137d7004c621c3e3e5"}
Sep 16 18:47:09.795559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.795477    2576 scope.go:117] "RemoveContainer" containerID="149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"
Sep 16 18:47:09.806971 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.806948    2576 scope.go:117] "RemoveContainer" containerID="149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"
Sep 16 18:47:09.807349 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:47:09.807315    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90\": container with ID starting with 149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90 not found: ID does not exist" containerID="149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"
Sep 16 18:47:09.807435 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.807354    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90"} err="failed to get container status \"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90\": rpc error: code = NotFound desc = could not find container \"149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90\": container with ID starting with 149679d3c5497604d5f520ffcb9cf696c04ca7cdfb3ae5be43db26eca5238d90 not found: ID does not exist"
Sep 16 18:47:09.831208 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.831180    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:47:09.838020 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:09.837998    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-16ae98-59d757947-wd448"]
Sep 16 18:47:11.011778 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:11.011745    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" path="/var/lib/kubelet/pods/64792d94-d250-4394-8fa7-b6e0e9ce0819/volumes"
Sep 16 18:47:12.786476 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:12.786448    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:47:32.117039 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.116961    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:47:32.117500 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.117483    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerName="maas-api"
Sep 16 18:47:32.117556 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.117504    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerName="maas-api"
Sep 16 18:47:32.117659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.117643    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="64792d94-d250-4394-8fa7-b6e0e9ce0819" containerName="maas-api"
Sep 16 18:47:32.121150 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.121133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.123417 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.123397    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d91d95b4-serving-cert\""
Sep 16 18:47:32.132122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.131899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:47:32.220533 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.220494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.220714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.220585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.220714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.220648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.220714 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.220671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8bjc\" (UniqueName: \"kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.321334 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.321297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.321510 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.321365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.321510 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.321419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.321510 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.321449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8bjc\" (UniqueName: \"kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.321675 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:47:32.321586    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-serving-cert: secret "maas-api-e2e-mt-d91d95b4-serving-cert" not found
Sep 16 18:47:32.321729 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:47:32.321682    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls podName:9dac91d0-4974-42ed-a021-70cf6b482ea8 nodeName:}" failed. No retries permitted until 2026-09-16 18:47:32.821662274 +0000 UTC m=+1180.242149115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls") pod "maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" (UID: "9dac91d0-4974-42ed-a021-70cf6b482ea8") : secret "maas-api-e2e-mt-d91d95b4-serving-cert" not found
Sep 16 18:47:32.322052 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.322033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.323923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.323904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.332541 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.332517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8bjc\" (UniqueName: \"kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.824478 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.824395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:32.826881 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:32.826855    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") pod \"maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:33.034472 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.034436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:33.319661 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.319626    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:47:33.321424 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:47:33.321394    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dac91d0_4974_42ed_a021_70cf6b482ea8.slice/crio-87d1c50cc9a6e161c3a701f7d9abc7da1b6c357d65d94d7d32547999e5d0ef50 WatchSource:0}: Error finding container 87d1c50cc9a6e161c3a701f7d9abc7da1b6c357d65d94d7d32547999e5d0ef50: Status 404 returned error can't find the container with id 87d1c50cc9a6e161c3a701f7d9abc7da1b6c357d65d94d7d32547999e5d0ef50
Sep 16 18:47:33.944611 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.944577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" event={"ID":"9dac91d0-4974-42ed-a021-70cf6b482ea8","Type":"ContainerStarted","Data":"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"}
Sep 16 18:47:33.944773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.944639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" event={"ID":"9dac91d0-4974-42ed-a021-70cf6b482ea8","Type":"ContainerStarted","Data":"87d1c50cc9a6e161c3a701f7d9abc7da1b6c357d65d94d7d32547999e5d0ef50"}
Sep 16 18:47:33.944773 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.944672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:33.963891 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:33.963848    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" podStartSLOduration=1.5254021039999999 podStartE2EDuration="1.963832803s" totalImagesPullingTime="438.430699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:33.962351062 +0000 UTC m=+1181.382837921" watchObservedRunningTime="2026-09-16 18:47:33.963832803 +0000 UTC m=+1181.384319662"
Sep 16 18:47:39.956138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:39.956110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:47:57.759397 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:57.759361    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:47:57.759838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:57.759635    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" podUID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" containerName="maas-api" containerID="cri-o://3d639f28e81dfa677e04a44077d7bc59ae91524367bd65e0765007b138a80a79" gracePeriod=30
Sep 16 18:47:57.832987 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:57.832956    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:47:57.833256 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:57.833206    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" podUID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" containerName="payload-processing" containerID="cri-o://ba15c3603879af0c945c98e93934ab46c4d5e1a7ea022c8c919123d55830cf5e" gracePeriod=30
Sep 16 18:47:58.098827 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.098799    2576 generic.go:356] "Generic (PLEG): container finished" podID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" containerID="ba15c3603879af0c945c98e93934ab46c4d5e1a7ea022c8c919123d55830cf5e" exitCode=0
Sep 16 18:47:58.098996 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.098867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" event={"ID":"47ae63d5-3ba9-406e-9bea-ef5c478e58b0","Type":"ContainerDied","Data":"ba15c3603879af0c945c98e93934ab46c4d5e1a7ea022c8c919123d55830cf5e"}
Sep 16 18:47:58.100494 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.100469    2576 generic.go:356] "Generic (PLEG): container finished" podID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" containerID="3d639f28e81dfa677e04a44077d7bc59ae91524367bd65e0765007b138a80a79" exitCode=0
Sep 16 18:47:58.100596 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.100509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" event={"ID":"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e","Type":"ContainerDied","Data":"3d639f28e81dfa677e04a44077d7bc59ae91524367bd65e0765007b138a80a79"}
Sep 16 18:47:58.100596 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.100517    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:47:58.100596 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.100542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj" event={"ID":"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e","Type":"ContainerDied","Data":"8132da5e403e1b65384a6b49d4968ea91d33d9e04b889f5d6edd739a6b97c126"}
Sep 16 18:47:58.100596 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.100585    2576 scope.go:117] "RemoveContainer" containerID="3d639f28e81dfa677e04a44077d7bc59ae91524367bd65e0765007b138a80a79"
Sep 16 18:47:58.140218 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.140192    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:47:58.165849 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.165825    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls\") pod \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") "
Sep 16 18:47:58.166043 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.165878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca\") pod \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") "
Sep 16 18:47:58.166043 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.165916    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs\") pod \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") "
Sep 16 18:47:58.166163 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.166089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkvg2\" (UniqueName: \"kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2\") pod \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\" (UID: \"55a6d3ca-21d8-4fe0-8df6-0c87bd19026e\") "
Sep 16 18:47:58.166270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.166249    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" (UID: "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:58.166820 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.166510    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.168304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.168274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" (UID: "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:58.168304 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.168286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" (UID: "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:58.168665 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.168640    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2" (OuterVolumeSpecName: "kube-api-access-fkvg2") pod "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" (UID: "55a6d3ca-21d8-4fe0-8df6-0c87bd19026e"). InnerVolumeSpecName "kube-api-access-fkvg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:58.266883 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.266856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume\") pod \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") "
Sep 16 18:47:58.267089 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.266969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r2rn\" (UniqueName: \"kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn\") pod \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\" (UID: \"47ae63d5-3ba9-406e-9bea-ef5c478e58b0\") "
Sep 16 18:47:58.267186 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.267173    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkvg2\" (UniqueName: \"kubernetes.io/projected/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-kube-api-access-fkvg2\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.267246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.267186    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.267246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.267195    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.267355 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.267242    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47ae63d5-3ba9-406e-9bea-ef5c478e58b0" (UID: "47ae63d5-3ba9-406e-9bea-ef5c478e58b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:58.269173 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.269148    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn" (OuterVolumeSpecName: "kube-api-access-9r2rn") pod "47ae63d5-3ba9-406e-9bea-ef5c478e58b0" (UID: "47ae63d5-3ba9-406e-9bea-ef5c478e58b0"). InnerVolumeSpecName "kube-api-access-9r2rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:58.368074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.367983    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r2rn\" (UniqueName: \"kubernetes.io/projected/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-kube-api-access-9r2rn\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.368074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:58.368023    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ae63d5-3ba9-406e-9bea-ef5c478e58b0-plugins-config-volume\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:47:59.107922 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.107848    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"
Sep 16 18:47:59.109372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.109350    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"
Sep 16 18:47:59.109516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.109348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7" event={"ID":"47ae63d5-3ba9-406e-9bea-ef5c478e58b0","Type":"ContainerDied","Data":"ad5cd8bd410bd9ef3ef4804651b81240d6ac34df473e7058bc388502429088e7"}
Sep 16 18:47:59.109516 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.109474    2576 scope.go:117] "RemoveContainer" containerID="ba15c3603879af0c945c98e93934ab46c4d5e1a7ea022c8c919123d55830cf5e"
Sep 16 18:47:59.129060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.129037    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:47:59.135664 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.135644    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f41de-7d6986f7db-vvgvj"]
Sep 16 18:47:59.151883 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.151857    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:47:59.154439 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:47:59.154412    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f41de-5d4b995b8f-t9nb7"]
Sep 16 18:48:01.011420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:01.011390    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" path="/var/lib/kubelet/pods/47ae63d5-3ba9-406e-9bea-ef5c478e58b0/volumes"
Sep 16 18:48:01.011796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:01.011782    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" path="/var/lib/kubelet/pods/55a6d3ca-21d8-4fe0-8df6-0c87bd19026e/volumes"
Sep 16 18:48:20.546413 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.546375    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:48:20.546902 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.546588    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" podUID="10e4b8ba-36ae-4111-a302-be6f290f297e" containerName="maas-api" containerID="cri-o://0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78" gracePeriod=30
Sep 16 18:48:20.799924 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.799865    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:48:20.877199 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.877166    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clfcp\" (UniqueName: \"kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp\") pod \"10e4b8ba-36ae-4111-a302-be6f290f297e\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") "
Sep 16 18:48:20.877383 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.877244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") pod \"10e4b8ba-36ae-4111-a302-be6f290f297e\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") "
Sep 16 18:48:20.877383 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.877289    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs\") pod \"10e4b8ba-36ae-4111-a302-be6f290f297e\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") "
Sep 16 18:48:20.877383 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.877374    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca\") pod \"10e4b8ba-36ae-4111-a302-be6f290f297e\" (UID: \"10e4b8ba-36ae-4111-a302-be6f290f297e\") "
Sep 16 18:48:20.877722 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.877686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10e4b8ba-36ae-4111-a302-be6f290f297e" (UID: "10e4b8ba-36ae-4111-a302-be6f290f297e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:20.879555 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.879519    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp" (OuterVolumeSpecName: "kube-api-access-clfcp") pod "10e4b8ba-36ae-4111-a302-be6f290f297e" (UID: "10e4b8ba-36ae-4111-a302-be6f290f297e"). InnerVolumeSpecName "kube-api-access-clfcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:20.879655 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.879545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10e4b8ba-36ae-4111-a302-be6f290f297e" (UID: "10e4b8ba-36ae-4111-a302-be6f290f297e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:20.879655 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.879585    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10e4b8ba-36ae-4111-a302-be6f290f297e" (UID: "10e4b8ba-36ae-4111-a302-be6f290f297e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:20.978993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.978963    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfcp\" (UniqueName: \"kubernetes.io/projected/10e4b8ba-36ae-4111-a302-be6f290f297e-kube-api-access-clfcp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:48:20.978993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.978988    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:48:20.978993 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.978997    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e4b8ba-36ae-4111-a302-be6f290f297e-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:48:20.979213 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:20.979005    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10e4b8ba-36ae-4111-a302-be6f290f297e-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:48:21.244154 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.244124    2576 generic.go:356] "Generic (PLEG): container finished" podID="10e4b8ba-36ae-4111-a302-be6f290f297e" containerID="0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78" exitCode=0
Sep 16 18:48:21.244321 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.244170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" event={"ID":"10e4b8ba-36ae-4111-a302-be6f290f297e","Type":"ContainerDied","Data":"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"}
Sep 16 18:48:21.244321 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.244174    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"
Sep 16 18:48:21.244321 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.244213    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b" event={"ID":"10e4b8ba-36ae-4111-a302-be6f290f297e","Type":"ContainerDied","Data":"16f1595b0bbebbb632ee337e260730006356e2d826977442798aabcd520fc5f8"}
Sep 16 18:48:21.244321 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.244233    2576 scope.go:117] "RemoveContainer" containerID="0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"
Sep 16 18:48:21.254105 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.254052    2576 scope.go:117] "RemoveContainer" containerID="0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"
Sep 16 18:48:21.254470 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:48:21.254437    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78\": container with ID starting with 0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78 not found: ID does not exist" containerID="0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"
Sep 16 18:48:21.254583 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.254478    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78"} err="failed to get container status \"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78\": rpc error: code = NotFound desc = could not find container \"0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78\": container with ID starting with 0de5dbc81a800ce0df3b4e94610cd33e68b9e8059d3230c02b06fa6fa1da9c78 not found: ID does not exist"
Sep 16 18:48:21.272349 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.272327    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:48:21.277199 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:21.277177    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e8510f-56b675dbbd-qbx5b"]
Sep 16 18:48:23.012986 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:23.012953    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e4b8ba-36ae-4111-a302-be6f290f297e" path="/var/lib/kubelet/pods/10e4b8ba-36ae-4111-a302-be6f290f297e/volumes"
Sep 16 18:48:43.135697 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.135666    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:48:43.136138 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" containerName="payload-processing"
Sep 16 18:48:43.136197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136142    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" containerName="payload-processing"
Sep 16 18:48:43.136197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136149    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" containerName="maas-api"
Sep 16 18:48:43.136197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136154    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" containerName="maas-api"
Sep 16 18:48:43.136197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136165    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e4b8ba-36ae-4111-a302-be6f290f297e" containerName="maas-api"
Sep 16 18:48:43.136197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136170    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e4b8ba-36ae-4111-a302-be6f290f297e" containerName="maas-api"
Sep 16 18:48:43.136369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136252    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="10e4b8ba-36ae-4111-a302-be6f290f297e" containerName="maas-api"
Sep 16 18:48:43.136369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136262    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ae63d5-3ba9-406e-9bea-ef5c478e58b0" containerName="payload-processing"
Sep 16 18:48:43.136369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.136269    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a6d3ca-21d8-4fe0-8df6-0c87bd19026e" containerName="maas-api"
Sep 16 18:48:43.140803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.140789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.144481 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.144455    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d91d95b4\"/\"kube-root-ca.crt\""
Sep 16 18:48:43.144605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.144455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d91d95b4\"/\"e2e-lifecycle-model-d91d95b4-kserve-self-signed-certs\""
Sep 16 18:48:43.144605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.144461    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d91d95b4\"/\"openshift-service-ca.crt\""
Sep 16 18:48:43.144605 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.144461    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d91d95b4\"/\"default-dockercfg-w4gps\""
Sep 16 18:48:43.150306 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.150285    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:48:43.286828 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.286798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twqxf\" (UniqueName: \"kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.287013 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.286861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.287013 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.286905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.287013 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.286989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.287188 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.287087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.287188 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.287109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.387822 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.387822 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.387822 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twqxf\" (UniqueName: \"kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.387984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.388090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388374 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.388162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.388374 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.388343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.390955 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.390915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.391141 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.391119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.402710 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.401747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twqxf\" (UniqueName: \"kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf\") pod \"e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") " pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.451798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.451757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:43.732319 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:43.732288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:48:43.733240 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:48:43.733209    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod761dc5c2_5e35_4c97_97b2_740e362a95ea.slice/crio-026acdf199abf09754051d4d2d7df17e73eff33bbc69ee5629ce18cd0b807837 WatchSource:0}: Error finding container 026acdf199abf09754051d4d2d7df17e73eff33bbc69ee5629ce18cd0b807837: Status 404 returned error can't find the container with id 026acdf199abf09754051d4d2d7df17e73eff33bbc69ee5629ce18cd0b807837
Sep 16 18:48:44.383990 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:44.383896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" event={"ID":"761dc5c2-5e35-4c97-97b2-740e362a95ea","Type":"ContainerStarted","Data":"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"}
Sep 16 18:48:44.383990 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:44.383964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" event={"ID":"761dc5c2-5e35-4c97-97b2-740e362a95ea","Type":"ContainerStarted","Data":"026acdf199abf09754051d4d2d7df17e73eff33bbc69ee5629ce18cd0b807837"}
Sep 16 18:48:44.404406 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:44.404363    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" podStartSLOduration=1.404347166 podStartE2EDuration="1.404347166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:44.402327174 +0000 UTC m=+1251.822814037" watchObservedRunningTime="2026-09-16 18:48:44.404347166 +0000 UTC m=+1251.824834026"
Sep 16 18:48:53.452803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:53.452779    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:53.453221 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:53.452817    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:53.465151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:53.465130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:48:54.455739 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:48:54.455714    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:49:13.984469 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:13.984438    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 18:49:13.989300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:13.989279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:13.991680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:13.991657    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-f9c9b8-serving-cert\""
Sep 16 18:49:13.998908 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:13.998885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 18:49:14.091526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.091494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.091805 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.091772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhqkf\" (UniqueName: \"kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.092070 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.091878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.092070 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.091954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.192497 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.192473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.192634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.192514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.192634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.192565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.192634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.192587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqkf\" (UniqueName: \"kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.192760 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:49:14.192693    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-serving-cert: secret "maas-api-e2e-shared-a-w3-f9c9b8-serving-cert" not found
Sep 16 18:49:14.192760 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:49:14.192752    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls podName:ee472f3a-f0dd-4e62-9b80-02f1cbddd67c nodeName:}" failed. No retries permitted until 2026-09-16 18:49:14.69273747 +0000 UTC m=+1282.113224308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls") pod "maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" (UID: "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c") : secret "maas-api-e2e-shared-a-w3-f9c9b8-serving-cert" not found
Sep 16 18:49:14.193300 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.193270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.195006 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.194981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.202040 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.202017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhqkf\" (UniqueName: \"kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.698172 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.698134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.701051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.701023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.902401 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.902364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:14.904206 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.904179    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 18:49:14.908144 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.908117    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:14.911072 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.911051    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-f9c9b8\""
Sep 16 18:49:14.911946 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.911914    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-f9c9b8-dockercfg-qlsdr\""
Sep 16 18:49:14.931876 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:14.931831    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 18:49:15.001744 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.001697    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd9bn\" (UniqueName: \"kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.002460 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.002433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.103128 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.103084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd9bn\" (UniqueName: \"kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.103314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.103172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.103840 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.103815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.112823 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.112791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd9bn\" (UniqueName: \"kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn\") pod \"payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.189919 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.189891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 18:49:15.191035 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:49:15.191013    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee472f3a_f0dd_4e62_9b80_02f1cbddd67c.slice/crio-2eabf2c06288b4fd1ed9c553e56d51b7459b605bdb2a7cc77f01878e805f170b WatchSource:0}: Error finding container 2eabf2c06288b4fd1ed9c553e56d51b7459b605bdb2a7cc77f01878e805f170b: Status 404 returned error can't find the container with id 2eabf2c06288b4fd1ed9c553e56d51b7459b605bdb2a7cc77f01878e805f170b
Sep 16 18:49:15.238171 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.238111    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:15.511492 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.511453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 18:49:15.513095 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:49:15.513068    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be3c742_3497_4926_9328_750a84ea7f5f.slice/crio-df303885e3c4ad0d7bccab711c1e3bd9d8bb380d1b6d60941cf74aa8eefdcb9d WatchSource:0}: Error finding container df303885e3c4ad0d7bccab711c1e3bd9d8bb380d1b6d60941cf74aa8eefdcb9d: Status 404 returned error can't find the container with id df303885e3c4ad0d7bccab711c1e3bd9d8bb380d1b6d60941cf74aa8eefdcb9d
Sep 16 18:49:15.580177 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.580152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" event={"ID":"6be3c742-3497-4926-9328-750a84ea7f5f","Type":"ContainerStarted","Data":"df303885e3c4ad0d7bccab711c1e3bd9d8bb380d1b6d60941cf74aa8eefdcb9d"}
Sep 16 18:49:15.581554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:15.581529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" event={"ID":"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c","Type":"ContainerStarted","Data":"2eabf2c06288b4fd1ed9c553e56d51b7459b605bdb2a7cc77f01878e805f170b"}
Sep 16 18:49:16.589107 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.589066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" event={"ID":"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c","Type":"ContainerStarted","Data":"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"}
Sep 16 18:49:16.589544 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.589139    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:16.590308 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.590280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" event={"ID":"6be3c742-3497-4926-9328-750a84ea7f5f","Type":"ContainerStarted","Data":"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"}
Sep 16 18:49:16.590395 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.590335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:16.616607 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.616559    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" podStartSLOduration=3.185816415 podStartE2EDuration="3.616543321s" totalImagesPullingTime="430.726906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:16.615800884 +0000 UTC m=+1284.036287755" watchObservedRunningTime="2026-09-16 18:49:16.616543321 +0000 UTC m=+1284.037030181"
Sep 16 18:49:16.644470 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:16.644424    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" podStartSLOduration=2.6444100390000003 podStartE2EDuration="2.644410039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:16.641698542 +0000 UTC m=+1284.062185419" watchObservedRunningTime="2026-09-16 18:49:16.644410039 +0000 UTC m=+1284.064896899"
Sep 16 18:49:22.601868 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:22.601839    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 18:49:27.598091 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:27.598061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 18:49:38.372626 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.372594    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 18:49:38.376686 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.376665    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.379191 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.379171    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-e65cc4-serving-cert\""
Sep 16 18:49:38.387888 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.387867    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 18:49:38.419344 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.419316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.419464 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.419401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mdd2\" (UniqueName: \"kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.419515 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.419478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.419552 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.419519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.520647 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.520612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.520792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.520682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.520792 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.520752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.520914 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.520844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mdd2\" (UniqueName: \"kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.521456 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.521436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.523152 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.523128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.523241 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.523183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.532297 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.532277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mdd2\" (UniqueName: \"kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2\") pod \"maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.690184 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.690159    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:38.968752 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:38.968726    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 18:49:38.970523 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:49:38.970492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e56038_727a_42b4_a30a_05390088ccbe.slice/crio-703a4518a9b2dda816ce916f09a94cf3e37fb88afcc09fc7d0035bf8ae01d7fc WatchSource:0}: Error finding container 703a4518a9b2dda816ce916f09a94cf3e37fb88afcc09fc7d0035bf8ae01d7fc: Status 404 returned error can't find the container with id 703a4518a9b2dda816ce916f09a94cf3e37fb88afcc09fc7d0035bf8ae01d7fc
Sep 16 18:49:39.728801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:39.728771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" event={"ID":"41e56038-727a-42b4-a30a-05390088ccbe","Type":"ContainerStarted","Data":"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"}
Sep 16 18:49:39.729244 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:39.728827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" event={"ID":"41e56038-727a-42b4-a30a-05390088ccbe","Type":"ContainerStarted","Data":"703a4518a9b2dda816ce916f09a94cf3e37fb88afcc09fc7d0035bf8ae01d7fc"}
Sep 16 18:49:39.729244 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:39.728860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:49:39.751274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:39.751227    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" podStartSLOduration=1.329854411 podStartE2EDuration="1.751211887s" totalImagesPullingTime="421.357476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:39.750053809 +0000 UTC m=+1307.170540666" watchObservedRunningTime="2026-09-16 18:49:39.751211887 +0000 UTC m=+1307.171698752"
Sep 16 18:49:45.741700 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:49:45.741670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 18:51:06.240598 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.240564    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:51:06.241526 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.241495    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" podUID="9dac91d0-4974-42ed-a021-70cf6b482ea8" containerName="maas-api" containerID="cri-o://b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808" gracePeriod=30
Sep 16 18:51:06.493567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.493445    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:51:06.571688 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.571657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8bjc\" (UniqueName: \"kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc\") pod \"9dac91d0-4974-42ed-a021-70cf6b482ea8\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") "
Sep 16 18:51:06.571877 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.571757    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") pod \"9dac91d0-4974-42ed-a021-70cf6b482ea8\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") "
Sep 16 18:51:06.571877 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.571805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs\") pod \"9dac91d0-4974-42ed-a021-70cf6b482ea8\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") "
Sep 16 18:51:06.571877 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.571829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca\") pod \"9dac91d0-4974-42ed-a021-70cf6b482ea8\" (UID: \"9dac91d0-4974-42ed-a021-70cf6b482ea8\") "
Sep 16 18:51:06.572249 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.572220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9dac91d0-4974-42ed-a021-70cf6b482ea8" (UID: "9dac91d0-4974-42ed-a021-70cf6b482ea8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:06.574019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.573989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc" (OuterVolumeSpecName: "kube-api-access-t8bjc") pod "9dac91d0-4974-42ed-a021-70cf6b482ea8" (UID: "9dac91d0-4974-42ed-a021-70cf6b482ea8"). InnerVolumeSpecName "kube-api-access-t8bjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:06.574019 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.573994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9dac91d0-4974-42ed-a021-70cf6b482ea8" (UID: "9dac91d0-4974-42ed-a021-70cf6b482ea8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:06.574197 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.574182    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9dac91d0-4974-42ed-a021-70cf6b482ea8" (UID: "9dac91d0-4974-42ed-a021-70cf6b482ea8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:06.672886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.672855    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.672886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.672883    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dac91d0-4974-42ed-a021-70cf6b482ea8-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.672886 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.672892    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dac91d0-4974-42ed-a021-70cf6b482ea8-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.673122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:06.672902    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8bjc\" (UniqueName: \"kubernetes.io/projected/9dac91d0-4974-42ed-a021-70cf6b482ea8-kube-api-access-t8bjc\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:07.275614 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.275528    2576 generic.go:356] "Generic (PLEG): container finished" podID="9dac91d0-4974-42ed-a021-70cf6b482ea8" containerID="b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808" exitCode=0
Sep 16 18:51:07.275614 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.275574    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"
Sep 16 18:51:07.276074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.275616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" event={"ID":"9dac91d0-4974-42ed-a021-70cf6b482ea8","Type":"ContainerDied","Data":"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"}
Sep 16 18:51:07.276074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.275669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv" event={"ID":"9dac91d0-4974-42ed-a021-70cf6b482ea8","Type":"ContainerDied","Data":"87d1c50cc9a6e161c3a701f7d9abc7da1b6c357d65d94d7d32547999e5d0ef50"}
Sep 16 18:51:07.276074 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.275692    2576 scope.go:117] "RemoveContainer" containerID="b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"
Sep 16 18:51:07.285636 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.285619    2576 scope.go:117] "RemoveContainer" containerID="b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"
Sep 16 18:51:07.285970 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:51:07.285915    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808\": container with ID starting with b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808 not found: ID does not exist" containerID="b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"
Sep 16 18:51:07.286030 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.285984    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808"} err="failed to get container status \"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808\": rpc error: code = NotFound desc = could not find container \"b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808\": container with ID starting with b4dd2426fa2ab7692db88a5ae4ced28a3583e6abaf0849ca0f1dbd0f597dd808 not found: ID does not exist"
Sep 16 18:51:07.294851 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.294823    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:51:07.298814 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:07.298792    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d91d95b4-7d75c8d7dd-5sslv"]
Sep 16 18:51:09.011839 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:09.011806    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dac91d0-4974-42ed-a021-70cf6b482ea8" path="/var/lib/kubelet/pods/9dac91d0-4974-42ed-a021-70cf6b482ea8/volumes"
Sep 16 18:51:14.944001 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:14.943963    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:51:29.964235 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:29.964190    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" podUID="761dc5c2-5e35-4c97-97b2-740e362a95ea" containerName="main" containerID="cri-o://351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4" gracePeriod=45
Sep 16 18:51:30.217134 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.217077    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:51:30.309688 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309664    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.309838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309696    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.309838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.309838 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309745    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.309970 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twqxf\" (UniqueName: \"kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.310025 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:30.310025 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309999    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home" (OuterVolumeSpecName: "home") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:30.310025 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.309999    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm\") pod \"761dc5c2-5e35-4c97-97b2-740e362a95ea\" (UID: \"761dc5c2-5e35-4c97-97b2-740e362a95ea\") "
Sep 16 18:51:30.310188 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.310027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:30.310579 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.310558    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.310646 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.310583    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.310646 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.310598    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.312125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.312098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm" (OuterVolumeSpecName: "dshm") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:30.312229 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.312134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:30.312272 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.312242    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf" (OuterVolumeSpecName: "kube-api-access-twqxf") pod "761dc5c2-5e35-4c97-97b2-740e362a95ea" (UID: "761dc5c2-5e35-4c97-97b2-740e362a95ea"). InnerVolumeSpecName "kube-api-access-twqxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:30.411573 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.411549    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twqxf\" (UniqueName: \"kubernetes.io/projected/761dc5c2-5e35-4c97-97b2-740e362a95ea-kube-api-access-twqxf\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.411573 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.411575    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/761dc5c2-5e35-4c97-97b2-740e362a95ea-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.411743 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.411590    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/761dc5c2-5e35-4c97-97b2-740e362a95ea-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:51:30.414432 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.414398    2576 generic.go:356] "Generic (PLEG): container finished" podID="761dc5c2-5e35-4c97-97b2-740e362a95ea" containerID="351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4" exitCode=0
Sep 16 18:51:30.414554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.414449    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"
Sep 16 18:51:30.414554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.414455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" event={"ID":"761dc5c2-5e35-4c97-97b2-740e362a95ea","Type":"ContainerDied","Data":"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"}
Sep 16 18:51:30.414554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.414490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d" event={"ID":"761dc5c2-5e35-4c97-97b2-740e362a95ea","Type":"ContainerDied","Data":"026acdf199abf09754051d4d2d7df17e73eff33bbc69ee5629ce18cd0b807837"}
Sep 16 18:51:30.414554 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.414508    2576 scope.go:117] "RemoveContainer" containerID="351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"
Sep 16 18:51:30.427454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.427438    2576 scope.go:117] "RemoveContainer" containerID="351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"
Sep 16 18:51:30.427750 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:51:30.427728    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4\": container with ID starting with 351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4 not found: ID does not exist" containerID="351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"
Sep 16 18:51:30.427845 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.427759    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4"} err="failed to get container status \"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4\": rpc error: code = NotFound desc = could not find container \"351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4\": container with ID starting with 351dc0d1332f2334751cca9554495892714e9110c6390aa9004839b7e6e806d4 not found: ID does not exist"
Sep 16 18:51:30.445169 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.445147    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:51:30.457952 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:30.457905    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d91d95b4/e2e-lifecycle-model-d91d95b4-kserve-d5f799f9f-h4w2d"]
Sep 16 18:51:31.012133 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:51:31.012099    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761dc5c2-5e35-4c97-97b2-740e362a95ea" path="/var/lib/kubelet/pods/761dc5c2-5e35-4c97-97b2-740e362a95ea/volumes"
Sep 16 18:52:24.904954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.904902    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:52:24.905604 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905584    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761dc5c2-5e35-4c97-97b2-740e362a95ea" containerName="main"
Sep 16 18:52:24.905711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905607    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="761dc5c2-5e35-4c97-97b2-740e362a95ea" containerName="main"
Sep 16 18:52:24.905711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905636    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dac91d0-4974-42ed-a021-70cf6b482ea8" containerName="maas-api"
Sep 16 18:52:24.905711 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905645    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dac91d0-4974-42ed-a021-70cf6b482ea8" containerName="maas-api"
Sep 16 18:52:24.905873 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905766    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dac91d0-4974-42ed-a021-70cf6b482ea8" containerName="maas-api"
Sep 16 18:52:24.905873 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.905782    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="761dc5c2-5e35-4c97-97b2-740e362a95ea" containerName="main"
Sep 16 18:52:24.909314 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.909294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:24.911585 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.911563    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"kube-root-ca.crt\""
Sep 16 18:52:24.911697 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.911585    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"openshift-service-ca.crt\""
Sep 16 18:52:24.912370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.912351    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"default-dockercfg-dmtfp\""
Sep 16 18:52:24.912472 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.912403    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"auth-test-model-e65cc4-kserve-self-signed-certs\""
Sep 16 18:52:24.918125 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:24.917907    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:52:25.011081 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.011252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvsff\" (UniqueName: \"kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.011252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.011252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.011252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.011454 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.011286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112110 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvsff\" (UniqueName: \"kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112146    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112261 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112520 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112566 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.112613 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.112588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.114614 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.114591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.114745 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.114705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.120759 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.120741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvsff\" (UniqueName: \"kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff\") pod \"auth-test-model-e65cc4-kserve-ff9699887-rcktn\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.220349 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.220325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:25.502853 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.502813    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:52:25.503818 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:52:25.503791    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36102065_6995_4d1c_991f_c77526189d0e.slice/crio-c9e58e798c8f3c0970b7324054c35185e8d61386dfbfd772319ec40b38e358eb WatchSource:0}: Error finding container c9e58e798c8f3c0970b7324054c35185e8d61386dfbfd772319ec40b38e358eb: Status 404 returned error can't find the container with id c9e58e798c8f3c0970b7324054c35185e8d61386dfbfd772319ec40b38e358eb
Sep 16 18:52:25.768954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.768853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" event={"ID":"36102065-6995-4d1c-991f-c77526189d0e","Type":"ContainerStarted","Data":"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"}
Sep 16 18:52:25.768954 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.768890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" event={"ID":"36102065-6995-4d1c-991f-c77526189d0e","Type":"ContainerStarted","Data":"c9e58e798c8f3c0970b7324054c35185e8d61386dfbfd772319ec40b38e358eb"}
Sep 16 18:52:25.787940 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:25.787892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" podStartSLOduration=1.787876544 podStartE2EDuration="1.787876544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:25.786380069 +0000 UTC m=+1473.206866930" watchObservedRunningTime="2026-09-16 18:52:25.787876544 +0000 UTC m=+1473.208363429"
Sep 16 18:52:35.220634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:35.220596    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:35.221150 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:35.220650    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:35.240353 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:35.240324    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:52:35.839551 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:52:35.839522    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:53:35.696659 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.696625    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:53:35.699943 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.699903    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.702794 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.702776    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-66b561-serving-cert\""
Sep 16 18:53:35.716617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.716581    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:53:35.800553 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.800527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9gv\" (UniqueName: \"kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.800692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.800566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.800692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.800606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.800692 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.800652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.901918 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.901887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9gv\" (UniqueName: \"kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.902096 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.901949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.902096 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.901995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.902096 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.902012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.902667 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.902641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.904847 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.904827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.904985 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.904835    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:35.910747 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:35.910723    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9gv\" (UniqueName: \"kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv\") pod \"maas-api-e2e-api-a-66b561-5646db559c-ww94r\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:36.012156 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:36.012094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:36.297688 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:36.297656    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:53:36.299128 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:53:36.299100    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073026ed_33cb_4a48_9668_d5eaad32deab.slice/crio-f99ec6d81f72d26e5c3e5d15e3fd31d426b2d175bdda2a13a17d885a552cbb45 WatchSource:0}: Error finding container f99ec6d81f72d26e5c3e5d15e3fd31d426b2d175bdda2a13a17d885a552cbb45: Status 404 returned error can't find the container with id f99ec6d81f72d26e5c3e5d15e3fd31d426b2d175bdda2a13a17d885a552cbb45
Sep 16 18:53:36.300595 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:36.300574    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:53:37.196059 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:37.196027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" event={"ID":"073026ed-33cb-4a48-9668-d5eaad32deab","Type":"ContainerStarted","Data":"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"}
Sep 16 18:53:37.196417 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:37.196076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" event={"ID":"073026ed-33cb-4a48-9668-d5eaad32deab","Type":"ContainerStarted","Data":"f99ec6d81f72d26e5c3e5d15e3fd31d426b2d175bdda2a13a17d885a552cbb45"}
Sep 16 18:53:37.196417 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:37.196126    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:53:37.215617 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:37.215573    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" podStartSLOduration=1.487107333 podStartE2EDuration="2.215558229s" totalImagesPullingTime="728.450896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:37.213025398 +0000 UTC m=+1544.633512258" watchObservedRunningTime="2026-09-16 18:53:37.215558229 +0000 UTC m=+1544.636045440"
Sep 16 18:53:43.208679 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:53:43.208655    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:54:01.015981 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.015948    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:54:01.019685 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.019664    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.022438 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.022386    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-41abff-serving-cert\""
Sep 16 18:54:01.029900 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.029665    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:54:01.149021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.148985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2zcx\" (UniqueName: \"kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.149190 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.149049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.149249 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.149190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.149412 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.149388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.250161 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.250125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.250323 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.250186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.250323 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.250241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.250426 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.250331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2zcx\" (UniqueName: \"kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.250917 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.250896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.252687 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.252660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.252789 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.252712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.259294 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.259267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2zcx\" (UniqueName: \"kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx\") pod \"maas-api-e2e-api-b-41abff-5898b4c496-bh8gv\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.335351 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.335285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:01.614361 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.614337    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:54:01.617464 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:54:01.617430    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4610cb10_c005_4362_8a42_4011779c5f36.slice/crio-dd444e3025b26d1e44f0170b7b1ea6d6931d1c89577e30add5fe2bc70f2aa48c WatchSource:0}: Error finding container dd444e3025b26d1e44f0170b7b1ea6d6931d1c89577e30add5fe2bc70f2aa48c: Status 404 returned error can't find the container with id dd444e3025b26d1e44f0170b7b1ea6d6931d1c89577e30add5fe2bc70f2aa48c
Sep 16 18:54:01.795064 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.795025    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:54:01.800050 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.800031    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:01.802635 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.802610    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-41abff\""
Sep 16 18:54:01.802756 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.802702    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-41abff-dockercfg-xj8kj\""
Sep 16 18:54:01.820610 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.820578    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:54:01.962811 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.957143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bpvj\" (UniqueName: \"kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:01.962811 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:01.957264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.058597 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.058563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.058964 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.058644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bpvj\" (UniqueName: \"kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.059262 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.059243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.069824 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.069796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bpvj\" (UniqueName: \"kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj\") pod \"payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") " pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.114358 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.114333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:02.361612 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.361574    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" event={"ID":"4610cb10-c005-4362-8a42-4011779c5f36","Type":"ContainerStarted","Data":"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"}
Sep 16 18:54:02.361750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.361632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" event={"ID":"4610cb10-c005-4362-8a42-4011779c5f36","Type":"ContainerStarted","Data":"dd444e3025b26d1e44f0170b7b1ea6d6931d1c89577e30add5fe2bc70f2aa48c"}
Sep 16 18:54:02.361750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.361734    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:02.386563 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.386523    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" podStartSLOduration=1.795363549 podStartE2EDuration="2.386507454s" totalImagesPullingTime="591.143905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:02.384261206 +0000 UTC m=+1569.804748079" watchObservedRunningTime="2026-09-16 18:54:02.386507454 +0000 UTC m=+1569.806994315"
Sep 16 18:54:02.405329 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:02.405302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:54:02.406387 ip-10-0-131-165 kubenswrapper[2576]: W0916 18:54:02.406363    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822c541c_30d7_4cd2_a369_3de5b5dd64e4.slice/crio-91cebca4593e21250f6f435c54db55d4bbc1daac1588fd4b97d20ffe1bfebca4 WatchSource:0}: Error finding container 91cebca4593e21250f6f435c54db55d4bbc1daac1588fd4b97d20ffe1bfebca4: Status 404 returned error can't find the container with id 91cebca4593e21250f6f435c54db55d4bbc1daac1588fd4b97d20ffe1bfebca4
Sep 16 18:54:03.369061 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:03.369027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" event={"ID":"822c541c-30d7-4cd2-a369-3de5b5dd64e4","Type":"ContainerStarted","Data":"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"}
Sep 16 18:54:03.369566 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:03.369096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:03.369566 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:03.369118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" event={"ID":"822c541c-30d7-4cd2-a369-3de5b5dd64e4","Type":"ContainerStarted","Data":"91cebca4593e21250f6f435c54db55d4bbc1daac1588fd4b97d20ffe1bfebca4"}
Sep 16 18:54:03.390057 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:03.390010    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" podStartSLOduration=2.389996117 podStartE2EDuration="2.389996117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:03.389565494 +0000 UTC m=+1570.810052358" watchObservedRunningTime="2026-09-16 18:54:03.389996117 +0000 UTC m=+1570.810482976"
Sep 16 18:54:08.375393 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:08.375365    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:54:14.377249 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:14.377221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:54:39.496925 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.496893    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:54:39.497530 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.497146    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" podUID="073026ed-33cb-4a48-9668-d5eaad32deab" containerName="maas-api" containerID="cri-o://9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403" gracePeriod=30
Sep 16 18:54:39.757712 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.757650    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:54:39.934041 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.934007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls\") pod \"073026ed-33cb-4a48-9668-d5eaad32deab\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") "
Sep 16 18:54:39.934234 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.934099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx9gv\" (UniqueName: \"kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv\") pod \"073026ed-33cb-4a48-9668-d5eaad32deab\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") "
Sep 16 18:54:39.934234 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.934153    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca\") pod \"073026ed-33cb-4a48-9668-d5eaad32deab\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") "
Sep 16 18:54:39.934234 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.934215    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs\") pod \"073026ed-33cb-4a48-9668-d5eaad32deab\" (UID: \"073026ed-33cb-4a48-9668-d5eaad32deab\") "
Sep 16 18:54:39.934567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.934539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "073026ed-33cb-4a48-9668-d5eaad32deab" (UID: "073026ed-33cb-4a48-9668-d5eaad32deab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:54:39.936506 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.936473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "073026ed-33cb-4a48-9668-d5eaad32deab" (UID: "073026ed-33cb-4a48-9668-d5eaad32deab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:54:39.936627 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.936554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv" (OuterVolumeSpecName: "kube-api-access-lx9gv") pod "073026ed-33cb-4a48-9668-d5eaad32deab" (UID: "073026ed-33cb-4a48-9668-d5eaad32deab"). InnerVolumeSpecName "kube-api-access-lx9gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:54:39.936696 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:39.936604    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "073026ed-33cb-4a48-9668-d5eaad32deab" (UID: "073026ed-33cb-4a48-9668-d5eaad32deab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:54:40.036274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.036175    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx9gv\" (UniqueName: \"kubernetes.io/projected/073026ed-33cb-4a48-9668-d5eaad32deab-kube-api-access-lx9gv\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:54:40.036274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.036213    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/073026ed-33cb-4a48-9668-d5eaad32deab-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:54:40.036274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.036229    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:54:40.036274 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.036241    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/073026ed-33cb-4a48-9668-d5eaad32deab-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:54:40.610166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.610131    2576 generic.go:356] "Generic (PLEG): container finished" podID="073026ed-33cb-4a48-9668-d5eaad32deab" containerID="9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403" exitCode=0
Sep 16 18:54:40.610166 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.610169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" event={"ID":"073026ed-33cb-4a48-9668-d5eaad32deab","Type":"ContainerDied","Data":"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"}
Sep 16 18:54:40.610671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.610174    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"
Sep 16 18:54:40.610671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.610223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r" event={"ID":"073026ed-33cb-4a48-9668-d5eaad32deab","Type":"ContainerDied","Data":"f99ec6d81f72d26e5c3e5d15e3fd31d426b2d175bdda2a13a17d885a552cbb45"}
Sep 16 18:54:40.610671 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.610255    2576 scope.go:117] "RemoveContainer" containerID="9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"
Sep 16 18:54:40.623676 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.623647    2576 scope.go:117] "RemoveContainer" containerID="9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"
Sep 16 18:54:40.624005 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:54:40.623983    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403\": container with ID starting with 9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403 not found: ID does not exist" containerID="9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"
Sep 16 18:54:40.624099 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.624014    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403"} err="failed to get container status \"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403\": rpc error: code = NotFound desc = could not find container \"9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403\": container with ID starting with 9ecabb2fcd870e9895197fb847dc6f7a5445f6f15cf8d222b6655168762f9403 not found: ID does not exist"
Sep 16 18:54:40.635372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.635345    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:54:40.641640 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:40.641616    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-66b561-5646db559c-ww94r"]
Sep 16 18:54:41.011705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:54:41.011674    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="073026ed-33cb-4a48-9668-d5eaad32deab" path="/var/lib/kubelet/pods/073026ed-33cb-4a48-9668-d5eaad32deab/volumes"
Sep 16 18:55:10.173843 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.173801    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:55:10.174460 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.174085    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" podUID="4610cb10-c005-4362-8a42-4011779c5f36" containerName="maas-api" containerID="cri-o://cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e" gracePeriod=30
Sep 16 18:55:10.230790 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.230754    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:55:10.231021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.230983    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" podUID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" containerName="payload-processing" containerID="cri-o://84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d" gracePeriod=30
Sep 16 18:55:10.494591 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.494567    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:55:10.526806 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.526780    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:55:10.634004 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.633963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs\") pod \"4610cb10-c005-4362-8a42-4011779c5f36\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") "
Sep 16 18:55:10.634222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bpvj\" (UniqueName: \"kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj\") pod \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") "
Sep 16 18:55:10.634222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2zcx\" (UniqueName: \"kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx\") pod \"4610cb10-c005-4362-8a42-4011779c5f36\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") "
Sep 16 18:55:10.634222 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634212    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume\") pod \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\" (UID: \"822c541c-30d7-4cd2-a369-3de5b5dd64e4\") "
Sep 16 18:55:10.634420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634277    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls\") pod \"4610cb10-c005-4362-8a42-4011779c5f36\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") "
Sep 16 18:55:10.634420 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634344    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca\") pod \"4610cb10-c005-4362-8a42-4011779c5f36\" (UID: \"4610cb10-c005-4362-8a42-4011779c5f36\") "
Sep 16 18:55:10.635116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634789    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "822c541c-30d7-4cd2-a369-3de5b5dd64e4" (UID: "822c541c-30d7-4cd2-a369-3de5b5dd64e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:10.635116 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.634996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4610cb10-c005-4362-8a42-4011779c5f36" (UID: "4610cb10-c005-4362-8a42-4011779c5f36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:10.636674 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.636643    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4610cb10-c005-4362-8a42-4011779c5f36" (UID: "4610cb10-c005-4362-8a42-4011779c5f36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:10.636803 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.636733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj" (OuterVolumeSpecName: "kube-api-access-6bpvj") pod "822c541c-30d7-4cd2-a369-3de5b5dd64e4" (UID: "822c541c-30d7-4cd2-a369-3de5b5dd64e4"). InnerVolumeSpecName "kube-api-access-6bpvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:10.637032 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.637005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx" (OuterVolumeSpecName: "kube-api-access-n2zcx") pod "4610cb10-c005-4362-8a42-4011779c5f36" (UID: "4610cb10-c005-4362-8a42-4011779c5f36"). InnerVolumeSpecName "kube-api-access-n2zcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:10.637140 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.637032    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4610cb10-c005-4362-8a42-4011779c5f36" (UID: "4610cb10-c005-4362-8a42-4011779c5f36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735488    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bpvj\" (UniqueName: \"kubernetes.io/projected/822c541c-30d7-4cd2-a369-3de5b5dd64e4-kube-api-access-6bpvj\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735516    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2zcx\" (UniqueName: \"kubernetes.io/projected/4610cb10-c005-4362-8a42-4011779c5f36-kube-api-access-n2zcx\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735525    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c541c-30d7-4cd2-a369-3de5b5dd64e4-plugins-config-volume\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735534    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735544    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4610cb10-c005-4362-8a42-4011779c5f36-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.735567 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.735553    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4610cb10-c005-4362-8a42-4011779c5f36-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:10.804017 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.803987    2576 generic.go:356] "Generic (PLEG): container finished" podID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" containerID="84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d" exitCode=0
Sep 16 18:55:10.804207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.804052    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"
Sep 16 18:55:10.804207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.804056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" event={"ID":"822c541c-30d7-4cd2-a369-3de5b5dd64e4","Type":"ContainerDied","Data":"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"}
Sep 16 18:55:10.804207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.804159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb" event={"ID":"822c541c-30d7-4cd2-a369-3de5b5dd64e4","Type":"ContainerDied","Data":"91cebca4593e21250f6f435c54db55d4bbc1daac1588fd4b97d20ffe1bfebca4"}
Sep 16 18:55:10.804207 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.804195    2576 scope.go:117] "RemoveContainer" containerID="84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"
Sep 16 18:55:10.805559 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.805536    2576 generic.go:356] "Generic (PLEG): container finished" podID="4610cb10-c005-4362-8a42-4011779c5f36" containerID="cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e" exitCode=0
Sep 16 18:55:10.805680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.805568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" event={"ID":"4610cb10-c005-4362-8a42-4011779c5f36","Type":"ContainerDied","Data":"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"}
Sep 16 18:55:10.805680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.805580    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"
Sep 16 18:55:10.805680 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.805602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv" event={"ID":"4610cb10-c005-4362-8a42-4011779c5f36","Type":"ContainerDied","Data":"dd444e3025b26d1e44f0170b7b1ea6d6931d1c89577e30add5fe2bc70f2aa48c"}
Sep 16 18:55:10.816001 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.815985    2576 scope.go:117] "RemoveContainer" containerID="84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"
Sep 16 18:55:10.816263 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:10.816246    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d\": container with ID starting with 84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d not found: ID does not exist" containerID="84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"
Sep 16 18:55:10.816307 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.816273    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d"} err="failed to get container status \"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d\": rpc error: code = NotFound desc = could not find container \"84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d\": container with ID starting with 84e0e0b42d0040df4e3cad9e2021356ff9f490686af5b32384649a85f671763d not found: ID does not exist"
Sep 16 18:55:10.816307 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.816289    2576 scope.go:117] "RemoveContainer" containerID="cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"
Sep 16 18:55:10.825705 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.825568    2576 scope.go:117] "RemoveContainer" containerID="cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"
Sep 16 18:55:10.826046 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:10.826019    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e\": container with ID starting with cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e not found: ID does not exist" containerID="cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"
Sep 16 18:55:10.826160 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.826057    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e"} err="failed to get container status \"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e\": rpc error: code = NotFound desc = could not find container \"cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e\": container with ID starting with cf25d0a67bdd7c8084498ba5ca00991bf45a37d5e059ddfc36dc36a35126a64e not found: ID does not exist"
Sep 16 18:55:10.828985 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.828963    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:55:10.833363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.833345    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-41abff-f5d88bcb9-nkzbb"]
Sep 16 18:55:10.846359 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.846330    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:55:10.850639 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:10.850616    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-41abff-5898b4c496-bh8gv"]
Sep 16 18:55:11.012092 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.012015    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4610cb10-c005-4362-8a42-4011779c5f36" path="/var/lib/kubelet/pods/4610cb10-c005-4362-8a42-4011779c5f36/volumes"
Sep 16 18:55:11.012427 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.012414    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" path="/var/lib/kubelet/pods/822c541c-30d7-4cd2-a369-3de5b5dd64e4/volumes"
Sep 16 18:55:11.166649 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.166622    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:55:11.166913 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.166883    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" podUID="5d248cfc-2730-4165-bec4-949bb7d62056" containerName="maas-api" containerID="cri-o://87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8" gracePeriod=30
Sep 16 18:55:11.420618 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.420583    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:55:11.543860 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.543777    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca\") pod \"5d248cfc-2730-4165-bec4-949bb7d62056\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") "
Sep 16 18:55:11.543860 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.543826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") pod \"5d248cfc-2730-4165-bec4-949bb7d62056\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") "
Sep 16 18:55:11.543860 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.543855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbkql\" (UniqueName: \"kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql\") pod \"5d248cfc-2730-4165-bec4-949bb7d62056\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") "
Sep 16 18:55:11.544146 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.543914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs\") pod \"5d248cfc-2730-4165-bec4-949bb7d62056\" (UID: \"5d248cfc-2730-4165-bec4-949bb7d62056\") "
Sep 16 18:55:11.544226 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.544185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d248cfc-2730-4165-bec4-949bb7d62056" (UID: "5d248cfc-2730-4165-bec4-949bb7d62056"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:11.544369 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.544351    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d248cfc-2730-4165-bec4-949bb7d62056-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:11.546431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.546394    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d248cfc-2730-4165-bec4-949bb7d62056" (UID: "5d248cfc-2730-4165-bec4-949bb7d62056"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:11.546713 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.546686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d248cfc-2730-4165-bec4-949bb7d62056" (UID: "5d248cfc-2730-4165-bec4-949bb7d62056"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:11.546796 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.546768    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql" (OuterVolumeSpecName: "kube-api-access-wbkql") pod "5d248cfc-2730-4165-bec4-949bb7d62056" (UID: "5d248cfc-2730-4165-bec4-949bb7d62056"). InnerVolumeSpecName "kube-api-access-wbkql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:11.645669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.645635    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:11.645669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.645668    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbkql\" (UniqueName: \"kubernetes.io/projected/5d248cfc-2730-4165-bec4-949bb7d62056-kube-api-access-wbkql\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:11.645887 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.645681    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d248cfc-2730-4165-bec4-949bb7d62056-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:11.813392 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.813311    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d248cfc-2730-4165-bec4-949bb7d62056" containerID="87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8" exitCode=0
Sep 16 18:55:11.813392 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.813355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"
Sep 16 18:55:11.813599 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.813397    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" event={"ID":"5d248cfc-2730-4165-bec4-949bb7d62056","Type":"ContainerDied","Data":"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"}
Sep 16 18:55:11.813599 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.813452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b" event={"ID":"5d248cfc-2730-4165-bec4-949bb7d62056","Type":"ContainerDied","Data":"d9bc2765f2370917552e03dd5a6573de1836e8936ed11b1ad3a5279a14fbe9fa"}
Sep 16 18:55:11.813599 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.813481    2576 scope.go:117] "RemoveContainer" containerID="87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"
Sep 16 18:55:11.824151 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.824137    2576 scope.go:117] "RemoveContainer" containerID="87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"
Sep 16 18:55:11.824407 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:11.824386    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8\": container with ID starting with 87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8 not found: ID does not exist" containerID="87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"
Sep 16 18:55:11.824467 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.824414    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8"} err="failed to get container status \"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8\": rpc error: code = NotFound desc = could not find container \"87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8\": container with ID starting with 87d1077cf141cd0f1468c765133c55a91eab4866bbe483f061efea34d37d3ab8 not found: ID does not exist"
Sep 16 18:55:11.837425 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.837406    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:55:11.842051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:11.842032    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4dda98-79bcdcd4c-2r45b"]
Sep 16 18:55:13.011250 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:13.011220    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d248cfc-2730-4165-bec4-949bb7d62056" path="/var/lib/kubelet/pods/5d248cfc-2730-4165-bec4-949bb7d62056/volumes"
Sep 16 18:55:19.614798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:19.613823    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:55:19.614798 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:19.614118    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy" containerID="cri-o://44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693" gracePeriod=30
Sep 16 18:55:24.881872 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.881846    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:55:24.898246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.898182    2576 generic.go:356] "Generic (PLEG): container finished" podID="42734aa2-125f-4cfe-847c-44d413960998" containerID="44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693" exitCode=0
Sep 16 18:55:24.898246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.898217    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerDied","Data":"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"}
Sep 16 18:55:24.898246 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.898235    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"
Sep 16 18:55:24.898459 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.898260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d" event={"ID":"42734aa2-125f-4cfe-847c-44d413960998","Type":"ContainerDied","Data":"5abfea37d6f048d97d1c49026faef4cd34b94db1feb078464f871dc7bf0e4728"}
Sep 16 18:55:24.898459 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.898282    2576 scope.go:117] "RemoveContainer" containerID="44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"
Sep 16 18:55:24.909135 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.909076    2576 scope.go:117] "RemoveContainer" containerID="8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"
Sep 16 18:55:24.918742 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.918726    2576 scope.go:117] "RemoveContainer" containerID="44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"
Sep 16 18:55:24.919076 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:24.919050    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693\": container with ID starting with 44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693 not found: ID does not exist" containerID="44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"
Sep 16 18:55:24.919144 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.919081    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693"} err="failed to get container status \"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693\": rpc error: code = NotFound desc = could not find container \"44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693\": container with ID starting with 44fc98679e37ef83052ed71baf67197d13c559dd8a2dacfaae9eb004b4bac693 not found: ID does not exist"
Sep 16 18:55:24.919144 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.919096    2576 scope.go:117] "RemoveContainer" containerID="8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"
Sep 16 18:55:24.919378 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:24.919361    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad\": container with ID starting with 8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad not found: ID does not exist" containerID="8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"
Sep 16 18:55:24.919422 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.919383    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad"} err="failed to get container status \"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad\": rpc error: code = NotFound desc = could not find container \"8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad\": container with ID starting with 8c5e863c29f8faf554fd8777f831396deae274d775fcda5231f9badf3b437cad not found: ID does not exist"
Sep 16 18:55:24.975183 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975276 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975319    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975372 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975342    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl27x\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975529 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975382    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975529 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975438    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975529 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data\") pod \"42734aa2-125f-4cfe-847c-44d413960998\" (UID: \"42734aa2-125f-4cfe-847c-44d413960998\") "
Sep 16 18:55:24.975678 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:24.975678 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:24.975786 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975750    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data" (OuterVolumeSpecName: "istio-data") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:24.975786 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975768    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975816    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975916    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-credential-socket\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975952    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42734aa2-125f-4cfe-847c-44d413960998-istiod-ca-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975967    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-data\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975981    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-socket\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:24.976086 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.975993    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-workload-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:24.977562 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.977540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:55:24.977669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.977573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token" (OuterVolumeSpecName: "istio-token") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:24.977669 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.977661    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x" (OuterVolumeSpecName: "kube-api-access-xl27x") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "kube-api-access-xl27x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:24.977755 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:24.977724    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42734aa2-125f-4cfe-847c-44d413960998" (UID: "42734aa2-125f-4cfe-847c-44d413960998"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:25.076584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.076559    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42734aa2-125f-4cfe-847c-44d413960998-istio-podinfo\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:25.076584 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.076583    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-istio-token\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:25.077070 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.076767    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl27x\" (UniqueName: \"kubernetes.io/projected/42734aa2-125f-4cfe-847c-44d413960998-kube-api-access-xl27x\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:25.077157 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.077100    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42734aa2-125f-4cfe-847c-44d413960998-istio-envoy\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:25.219076 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.219046    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:55:25.227812 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:25.227789    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4dda98-openshift-default-68fcd4ccdd-v778d"]
Sep 16 18:55:27.012408 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:27.012377    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42734aa2-125f-4cfe-847c-44d413960998" path="/var/lib/kubelet/pods/42734aa2-125f-4cfe-847c-44d413960998/volumes"
Sep 16 18:55:33.586382 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.586343    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:55:33.586793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.586597    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" podUID="54a3607c-3f68-4407-87a7-82ab8206d7b4" containerName="maas-api" containerID="cri-o://bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e" gracePeriod=30
Sep 16 18:55:33.848921 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.848851    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:55:33.957055 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.957018    2576 generic.go:356] "Generic (PLEG): container finished" podID="54a3607c-3f68-4407-87a7-82ab8206d7b4" containerID="bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e" exitCode=0
Sep 16 18:55:33.957214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.957069    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"
Sep 16 18:55:33.957214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.957106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" event={"ID":"54a3607c-3f68-4407-87a7-82ab8206d7b4","Type":"ContainerDied","Data":"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"}
Sep 16 18:55:33.957214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.957161    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj" event={"ID":"54a3607c-3f68-4407-87a7-82ab8206d7b4","Type":"ContainerDied","Data":"b9eccc098def2f39c08ca64acc3641069efd5b5d0ce0f0dbfe4f7a643d799892"}
Sep 16 18:55:33.957214 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.957184    2576 scope.go:117] "RemoveContainer" containerID="bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"
Sep 16 18:55:33.959529 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.959505    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca\") pod \"54a3607c-3f68-4407-87a7-82ab8206d7b4\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") "
Sep 16 18:55:33.959684 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.959669    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs\") pod \"54a3607c-3f68-4407-87a7-82ab8206d7b4\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") "
Sep 16 18:55:33.959750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.959700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") pod \"54a3607c-3f68-4407-87a7-82ab8206d7b4\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") "
Sep 16 18:55:33.959750 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.959728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fbzb\" (UniqueName: \"kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb\") pod \"54a3607c-3f68-4407-87a7-82ab8206d7b4\" (UID: \"54a3607c-3f68-4407-87a7-82ab8206d7b4\") "
Sep 16 18:55:33.959889 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.959856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "54a3607c-3f68-4407-87a7-82ab8206d7b4" (UID: "54a3607c-3f68-4407-87a7-82ab8206d7b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:33.960170 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.960150    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a3607c-3f68-4407-87a7-82ab8206d7b4-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:33.962058 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.962037    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb" (OuterVolumeSpecName: "kube-api-access-4fbzb") pod "54a3607c-3f68-4407-87a7-82ab8206d7b4" (UID: "54a3607c-3f68-4407-87a7-82ab8206d7b4"). InnerVolumeSpecName "kube-api-access-4fbzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:33.962252 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.962232    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "54a3607c-3f68-4407-87a7-82ab8206d7b4" (UID: "54a3607c-3f68-4407-87a7-82ab8206d7b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:33.962387 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.962368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "54a3607c-3f68-4407-87a7-82ab8206d7b4" (UID: "54a3607c-3f68-4407-87a7-82ab8206d7b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:33.983377 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.983344    2576 scope.go:117] "RemoveContainer" containerID="bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"
Sep 16 18:55:33.983683 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:55:33.983662    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e\": container with ID starting with bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e not found: ID does not exist" containerID="bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"
Sep 16 18:55:33.983784 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:33.983694    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e"} err="failed to get container status \"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e\": rpc error: code = NotFound desc = could not find container \"bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e\": container with ID starting with bcbbca40bc47652644a3185d05297690b50c2fce5eee8a43b5938798533e5f8e not found: ID does not exist"
Sep 16 18:55:34.061685 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:34.061655    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:34.061685 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:34.061680    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a3607c-3f68-4407-87a7-82ab8206d7b4-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:34.061685 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:34.061689    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fbzb\" (UniqueName: \"kubernetes.io/projected/54a3607c-3f68-4407-87a7-82ab8206d7b4-kube-api-access-4fbzb\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:34.294051 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:34.294024    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:55:34.302524 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:34.302498    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ee98a5-64dd48fc47-bj5pj"]
Sep 16 18:55:35.011639 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:35.011609    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a3607c-3f68-4407-87a7-82ab8206d7b4" path="/var/lib/kubelet/pods/54a3607c-3f68-4407-87a7-82ab8206d7b4/volumes"
Sep 16 18:55:53.828553 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:53.828520    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:55:53.829060 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:53.828765    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" podUID="68637031-da63-464b-8de2-d455cf5f3d9e" containerName="maas-api" containerID="cri-o://0b07daf96824c55dccbbdd20c6d64f4dc8007e0ea0e377b0a14b76b46088b23b" gracePeriod=30
Sep 16 18:55:54.078801 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.078725    2576 generic.go:356] "Generic (PLEG): container finished" podID="68637031-da63-464b-8de2-d455cf5f3d9e" containerID="0b07daf96824c55dccbbdd20c6d64f4dc8007e0ea0e377b0a14b76b46088b23b" exitCode=0
Sep 16 18:55:54.078923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.078796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" event={"ID":"68637031-da63-464b-8de2-d455cf5f3d9e","Type":"ContainerDied","Data":"0b07daf96824c55dccbbdd20c6d64f4dc8007e0ea0e377b0a14b76b46088b23b"}
Sep 16 18:55:54.078923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.078845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7" event={"ID":"68637031-da63-464b-8de2-d455cf5f3d9e","Type":"ContainerDied","Data":"538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c"}
Sep 16 18:55:54.078923 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.078860    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="538c4e7e960cc10cca2ae770a0ad7e3b150d75da9e45359979ce1c258e3d978c"
Sep 16 18:55:54.084275 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.084254    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:55:54.252815 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.252775    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs\") pod \"68637031-da63-464b-8de2-d455cf5f3d9e\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") "
Sep 16 18:55:54.253021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.252838    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb9rm\" (UniqueName: \"kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm\") pod \"68637031-da63-464b-8de2-d455cf5f3d9e\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") "
Sep 16 18:55:54.253021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.252862    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca\") pod \"68637031-da63-464b-8de2-d455cf5f3d9e\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") "
Sep 16 18:55:54.253021 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.252983    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls\") pod \"68637031-da63-464b-8de2-d455cf5f3d9e\" (UID: \"68637031-da63-464b-8de2-d455cf5f3d9e\") "
Sep 16 18:55:54.253240 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.253217    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68637031-da63-464b-8de2-d455cf5f3d9e" (UID: "68637031-da63-464b-8de2-d455cf5f3d9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:54.253431 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.253416    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68637031-da63-464b-8de2-d455cf5f3d9e-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:54.255609 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.255575    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "68637031-da63-464b-8de2-d455cf5f3d9e" (UID: "68637031-da63-464b-8de2-d455cf5f3d9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:54.255800 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.255680    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68637031-da63-464b-8de2-d455cf5f3d9e" (UID: "68637031-da63-464b-8de2-d455cf5f3d9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:54.255800 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.255693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm" (OuterVolumeSpecName: "kube-api-access-bb9rm") pod "68637031-da63-464b-8de2-d455cf5f3d9e" (UID: "68637031-da63-464b-8de2-d455cf5f3d9e"). InnerVolumeSpecName "kube-api-access-bb9rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:54.354519 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.354449    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:54.354519 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.354474    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb9rm\" (UniqueName: \"kubernetes.io/projected/68637031-da63-464b-8de2-d455cf5f3d9e-kube-api-access-bb9rm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:54.354519 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:54.354484    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68637031-da63-464b-8de2-d455cf5f3d9e-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:55:55.085804 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:55.085774    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"
Sep 16 18:55:55.105084 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:55.105053    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:55:55.109651 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:55.109629    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8737d2-7f8f46dfd8-dskc7"]
Sep 16 18:55:57.011740 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:55:57.011708    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68637031-da63-464b-8de2-d455cf5f3d9e" path="/var/lib/kubelet/pods/68637031-da63-464b-8de2-d455cf5f3d9e/volumes"
Sep 16 18:56:53.285520 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:56:53.285427    2576 scope.go:117] "RemoveContainer" containerID="0b07daf96824c55dccbbdd20c6d64f4dc8007e0ea0e377b0a14b76b46088b23b"
Sep 16 18:57:25.128122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:25.128087    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:57:40.147639 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.147600    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" podUID="36102065-6995-4d1c-991f-c77526189d0e" containerName="main" containerID="cri-o://63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a" gracePeriod=45
Sep 16 18:57:40.403100 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.403038    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:57:40.501122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501085    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501122 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvsff\" (UniqueName: \"kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501363 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501266    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm\") pod \"36102065-6995-4d1c-991f-c77526189d0e\" (UID: \"36102065-6995-4d1c-991f-c77526189d0e\") "
Sep 16 18:57:40.501556 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501375    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:40.501556 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501465    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home" (OuterVolumeSpecName: "home") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:40.501556 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501482    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache" (OuterVolumeSpecName: "model-cache") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:40.501658 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501622    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.501658 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501640    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.501722 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.501669    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.503482 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.503458    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:57:40.503609 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.503589    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff" (OuterVolumeSpecName: "kube-api-access-gvsff") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "kube-api-access-gvsff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:40.503649 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.503626    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm" (OuterVolumeSpecName: "dshm") pod "36102065-6995-4d1c-991f-c77526189d0e" (UID: "36102065-6995-4d1c-991f-c77526189d0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:40.602793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.602764    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvsff\" (UniqueName: \"kubernetes.io/projected/36102065-6995-4d1c-991f-c77526189d0e-kube-api-access-gvsff\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.602793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.602789    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36102065-6995-4d1c-991f-c77526189d0e-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.602793 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.602798    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36102065-6995-4d1c-991f-c77526189d0e-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 18:57:40.712098 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.712063    2576 generic.go:356] "Generic (PLEG): container finished" podID="36102065-6995-4d1c-991f-c77526189d0e" containerID="63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a" exitCode=0
Sep 16 18:57:40.712270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.712108    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"
Sep 16 18:57:40.712270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.712154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" event={"ID":"36102065-6995-4d1c-991f-c77526189d0e","Type":"ContainerDied","Data":"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"}
Sep 16 18:57:40.712270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.712200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn" event={"ID":"36102065-6995-4d1c-991f-c77526189d0e","Type":"ContainerDied","Data":"c9e58e798c8f3c0970b7324054c35185e8d61386dfbfd772319ec40b38e358eb"}
Sep 16 18:57:40.712270 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.712217    2576 scope.go:117] "RemoveContainer" containerID="63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"
Sep 16 18:57:40.721841 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.721825    2576 scope.go:117] "RemoveContainer" containerID="63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"
Sep 16 18:57:40.722140 ip-10-0-131-165 kubenswrapper[2576]: E0916 18:57:40.722124    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a\": container with ID starting with 63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a not found: ID does not exist" containerID="63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"
Sep 16 18:57:40.722203 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.722148    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a"} err="failed to get container status \"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a\": rpc error: code = NotFound desc = could not find container \"63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a\": container with ID starting with 63f99ad7959705dc584995963a04bb1114531ed6d5f77e03ce1892836af6f24a not found: ID does not exist"
Sep 16 18:57:40.749775 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.749739    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:57:40.760634 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:40.760600    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/auth-test-model-e65cc4-kserve-ff9699887-rcktn"]
Sep 16 18:57:41.011370 ip-10-0-131-165 kubenswrapper[2576]: I0916 18:57:41.011291    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36102065-6995-4d1c-991f-c77526189d0e" path="/var/lib/kubelet/pods/36102065-6995-4d1c-991f-c77526189d0e/volumes"
Sep 16 19:01:10.222675 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.222638    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223109    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073026ed-33cb-4a48-9668-d5eaad32deab" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223121    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="073026ed-33cb-4a48-9668-d5eaad32deab" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223135    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68637031-da63-464b-8de2-d455cf5f3d9e" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223141    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68637031-da63-464b-8de2-d455cf5f3d9e" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223150    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4610cb10-c005-4362-8a42-4011779c5f36" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223157    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4610cb10-c005-4362-8a42-4011779c5f36" containerName="maas-api"
Sep 16 19:01:10.223165 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223169    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223174    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223182    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36102065-6995-4d1c-991f-c77526189d0e" containerName="main"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223190    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="36102065-6995-4d1c-991f-c77526189d0e" containerName="main"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223202    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d248cfc-2730-4165-bec4-949bb7d62056" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223210    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d248cfc-2730-4165-bec4-949bb7d62056" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223217    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" containerName="payload-processing"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223222    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" containerName="payload-processing"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223234    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a3607c-3f68-4407-87a7-82ab8206d7b4" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223239    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a3607c-3f68-4407-87a7-82ab8206d7b4" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223245    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223252    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223322    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223328    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4610cb10-c005-4362-8a42-4011779c5f36" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223334    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68637031-da63-464b-8de2-d455cf5f3d9e" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223341    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="073026ed-33cb-4a48-9668-d5eaad32deab" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223351    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c541c-30d7-4cd2-a369-3de5b5dd64e4" containerName="payload-processing"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223360    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d248cfc-2730-4165-bec4-949bb7d62056" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223365    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a3607c-3f68-4407-87a7-82ab8206d7b4" containerName="maas-api"
Sep 16 19:01:10.223388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.223371    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="36102065-6995-4d1c-991f-c77526189d0e" containerName="main"
Sep 16 19:01:10.226334 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.226319    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.236911 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.236888    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6a532e-serving-cert\""
Sep 16 19:01:10.253913 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.253884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:01:10.345132 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.345097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.345336 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.345143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.345336 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.345237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.345336 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.345284    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrtnr\" (UniqueName: \"kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.446153 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.446108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.446332 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.446171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.446332 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.446230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.446332 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.446260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrtnr\" (UniqueName: \"kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.446915 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.446894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.448959 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.448906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.449119 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.449099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.470595 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.470573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrtnr\" (UniqueName: \"kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr\") pod \"maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.537228 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.537162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:10.833411 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.833387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:01:10.835922 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:01:10.835896    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43702410_7574_4bb1_b041_a2d85720be26.slice/crio-3570fe634e5936a5ec903dee23bd24c13e42f3166afbeb031496ebd41e9570e4 WatchSource:0}: Error finding container 3570fe634e5936a5ec903dee23bd24c13e42f3166afbeb031496ebd41e9570e4: Status 404 returned error can't find the container with id 3570fe634e5936a5ec903dee23bd24c13e42f3166afbeb031496ebd41e9570e4
Sep 16 19:01:10.837276 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.837259    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:01:10.934799 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:10.934767    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" event={"ID":"43702410-7574-4bb1-b041-a2d85720be26","Type":"ContainerStarted","Data":"3570fe634e5936a5ec903dee23bd24c13e42f3166afbeb031496ebd41e9570e4"}
Sep 16 19:01:12.948269 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:12.948230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" event={"ID":"43702410-7574-4bb1-b041-a2d85720be26","Type":"ContainerStarted","Data":"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"}
Sep 16 19:01:12.948698 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:12.948302    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:12.984307 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:12.984258    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" podStartSLOduration=1.762601241 podStartE2EDuration="2.984244593s" totalImagesPullingTime="1.221643352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:12.980517904 +0000 UTC m=+2000.401004763" watchObservedRunningTime="2026-09-16 19:01:12.984244593 +0000 UTC m=+2000.404731453"
Sep 16 19:01:18.959907 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:18.959879    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:01:34.120138 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.120104    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:01:34.120644 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.120631    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42734aa2-125f-4cfe-847c-44d413960998" containerName="istio-proxy"
Sep 16 19:01:34.123904 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.123889    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.126864 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.126843    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a0f152-serving-cert\""
Sep 16 19:01:34.151311 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.151288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:01:34.163587 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.163559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.163723 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.163620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.163791 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.163723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm8zj\" (UniqueName: \"kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.163866 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.163847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.264985 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.264923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm8zj\" (UniqueName: \"kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.265122 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.265075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.265122 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.265107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.265201 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.265149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.265350 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:01:34.265326    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-serving-cert: secret "maas-api-e2e-ipp-b-a0f152-serving-cert" not found
Sep 16 19:01:34.265434 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:01:34.265422    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls podName:0a45707c-7cc5-4d04-9f94-b8f5247aac24 nodeName:}" failed. No retries permitted until 2026-09-16 19:01:34.765399177 +0000 UTC m=+2022.185886035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls") pod "maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" (UID: "0a45707c-7cc5-4d04-9f94-b8f5247aac24") : secret "maas-api-e2e-ipp-b-a0f152-serving-cert" not found
Sep 16 19:01:34.265833 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.265815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.267722 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.267701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.288763 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.288741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm8zj\" (UniqueName: \"kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.769809 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.769779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:34.772391 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:34.772360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:35.034162 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:35.034085    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:35.333287 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:01:35.333252    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a45707c_7cc5_4d04_9f94_b8f5247aac24.slice/crio-56bf1f821a110b7b48c8cee71afe3e7f178a22e6aad05e6793fee32a9484a767 WatchSource:0}: Error finding container 56bf1f821a110b7b48c8cee71afe3e7f178a22e6aad05e6793fee32a9484a767: Status 404 returned error can't find the container with id 56bf1f821a110b7b48c8cee71afe3e7f178a22e6aad05e6793fee32a9484a767
Sep 16 19:01:35.336924 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:35.336895    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:01:36.084481 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:36.084394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" event={"ID":"0a45707c-7cc5-4d04-9f94-b8f5247aac24","Type":"ContainerStarted","Data":"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"}
Sep 16 19:01:36.084481 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:36.084450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" event={"ID":"0a45707c-7cc5-4d04-9f94-b8f5247aac24","Type":"ContainerStarted","Data":"56bf1f821a110b7b48c8cee71afe3e7f178a22e6aad05e6793fee32a9484a767"}
Sep 16 19:01:36.084481 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:36.084472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:36.123408 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:36.123358    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" podStartSLOduration=1.760996271 podStartE2EDuration="2.123343624s" totalImagesPullingTime="362.347353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:36.120004594 +0000 UTC m=+2023.540491454" watchObservedRunningTime="2026-09-16 19:01:36.123343624 +0000 UTC m=+2023.543830484"
Sep 16 19:01:42.095537 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:42.095506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:01:55.526676 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:01:55.526644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:02:57.376862 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:02:57.376834    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:03:05.387522 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.387489    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:05.391036 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.391018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.393390 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.393368    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-0e536c-serving-cert\""
Sep 16 19:03:05.407299 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.407274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:05.482961 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.482922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxp6c\" (UniqueName: \"kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.483078 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.482972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.483121 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.483079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.483158 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.483130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.584331 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.584301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxp6c\" (UniqueName: \"kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.584331 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.584338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.584510 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.584413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.584510 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.584461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.585017 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.584990    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.587098 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.587077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.587183 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.587122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.594183 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.594163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxp6c\" (UniqueName: \"kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c\") pod \"maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.701449 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.701430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:05.979976 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:05.979948    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:05.981208 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:03:05.981181    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1663fef6_0100_4e77_a094_c722bb0a4b2f.slice/crio-5db5df8f003f97fc8c0ece589a4f0c9ac42b43c36ee7b06bd9f8f5ce901bbcb8 WatchSource:0}: Error finding container 5db5df8f003f97fc8c0ece589a4f0c9ac42b43c36ee7b06bd9f8f5ce901bbcb8: Status 404 returned error can't find the container with id 5db5df8f003f97fc8c0ece589a4f0c9ac42b43c36ee7b06bd9f8f5ce901bbcb8
Sep 16 19:03:06.614473 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:06.614441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" event={"ID":"1663fef6-0100-4e77-a094-c722bb0a4b2f","Type":"ContainerStarted","Data":"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"}
Sep 16 19:03:06.614850 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:06.614497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" event={"ID":"1663fef6-0100-4e77-a094-c722bb0a4b2f","Type":"ContainerStarted","Data":"5db5df8f003f97fc8c0ece589a4f0c9ac42b43c36ee7b06bd9f8f5ce901bbcb8"}
Sep 16 19:03:06.614850 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:06.614541    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:06.639952 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:06.639841    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" podStartSLOduration=1.255986782 podStartE2EDuration="1.639826151s" totalImagesPullingTime="383.839369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:03:06.637785137 +0000 UTC m=+2114.058272009" watchObservedRunningTime="2026-09-16 19:03:06.639826151 +0000 UTC m=+2114.060313010"
Sep 16 19:03:12.629203 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:12.629170    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:33.080098 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.080059    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:33.080581 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.080300    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" podUID="1663fef6-0100-4e77-a094-c722bb0a4b2f" containerName="maas-api" containerID="cri-o://722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948" gracePeriod=30
Sep 16 19:03:33.327373 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.327343    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:33.429311 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429277    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca\") pod \"1663fef6-0100-4e77-a094-c722bb0a4b2f\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") "
Sep 16 19:03:33.429484 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429341    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls\") pod \"1663fef6-0100-4e77-a094-c722bb0a4b2f\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") "
Sep 16 19:03:33.429484 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429363    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxp6c\" (UniqueName: \"kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c\") pod \"1663fef6-0100-4e77-a094-c722bb0a4b2f\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") "
Sep 16 19:03:33.429484 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429436    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs\") pod \"1663fef6-0100-4e77-a094-c722bb0a4b2f\" (UID: \"1663fef6-0100-4e77-a094-c722bb0a4b2f\") "
Sep 16 19:03:33.429649 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429621    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1663fef6-0100-4e77-a094-c722bb0a4b2f" (UID: "1663fef6-0100-4e77-a094-c722bb0a4b2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:03:33.429817 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.429798    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1663fef6-0100-4e77-a094-c722bb0a4b2f-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:03:33.431666 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.431643    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1663fef6-0100-4e77-a094-c722bb0a4b2f" (UID: "1663fef6-0100-4e77-a094-c722bb0a4b2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:03:33.431778 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.431684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c" (OuterVolumeSpecName: "kube-api-access-sxp6c") pod "1663fef6-0100-4e77-a094-c722bb0a4b2f" (UID: "1663fef6-0100-4e77-a094-c722bb0a4b2f"). InnerVolumeSpecName "kube-api-access-sxp6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:03:33.431778 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.431684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1663fef6-0100-4e77-a094-c722bb0a4b2f" (UID: "1663fef6-0100-4e77-a094-c722bb0a4b2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:03:33.531114 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.531084    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:03:33.531114 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.531110    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxp6c\" (UniqueName: \"kubernetes.io/projected/1663fef6-0100-4e77-a094-c722bb0a4b2f-kube-api-access-sxp6c\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:03:33.531114 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.531119    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1663fef6-0100-4e77-a094-c722bb0a4b2f-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:03:33.784213 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.784124    2576 generic.go:356] "Generic (PLEG): container finished" podID="1663fef6-0100-4e77-a094-c722bb0a4b2f" containerID="722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948" exitCode=0
Sep 16 19:03:33.784213 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.784165    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"
Sep 16 19:03:33.784213 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.784174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" event={"ID":"1663fef6-0100-4e77-a094-c722bb0a4b2f","Type":"ContainerDied","Data":"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"}
Sep 16 19:03:33.784445 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.784221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9" event={"ID":"1663fef6-0100-4e77-a094-c722bb0a4b2f","Type":"ContainerDied","Data":"5db5df8f003f97fc8c0ece589a4f0c9ac42b43c36ee7b06bd9f8f5ce901bbcb8"}
Sep 16 19:03:33.784445 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.784247    2576 scope.go:117] "RemoveContainer" containerID="722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"
Sep 16 19:03:33.794670 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.794652    2576 scope.go:117] "RemoveContainer" containerID="722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"
Sep 16 19:03:33.794967 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:03:33.794921    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948\": container with ID starting with 722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948 not found: ID does not exist" containerID="722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"
Sep 16 19:03:33.795027 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.794976    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948"} err="failed to get container status \"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948\": rpc error: code = NotFound desc = could not find container \"722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948\": container with ID starting with 722774aa347ceff2f5456e72d9ac71f56cfd062609c628bb7fc8d263569a5948 not found: ID does not exist"
Sep 16 19:03:33.807650 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.807627    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:33.815017 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:33.814998    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e536c-6df4654d6f-dx5m9"]
Sep 16 19:03:35.011815 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:03:35.011778    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1663fef6-0100-4e77-a094-c722bb0a4b2f" path="/var/lib/kubelet/pods/1663fef6-0100-4e77-a094-c722bb0a4b2f/volumes"
Sep 16 19:04:08.028146 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.028113    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:04:08.028630 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.028332    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" podUID="43702410-7574-4bb1-b041-a2d85720be26" containerName="maas-api" containerID="cri-o://4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2" gracePeriod=30
Sep 16 19:04:08.289984 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.289897    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:04:08.345814 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.345785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs\") pod \"43702410-7574-4bb1-b041-a2d85720be26\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") "
Sep 16 19:04:08.346015 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.345877    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrtnr\" (UniqueName: \"kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr\") pod \"43702410-7574-4bb1-b041-a2d85720be26\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") "
Sep 16 19:04:08.346015 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.345922    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls\") pod \"43702410-7574-4bb1-b041-a2d85720be26\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") "
Sep 16 19:04:08.346015 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.345987    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca\") pod \"43702410-7574-4bb1-b041-a2d85720be26\" (UID: \"43702410-7574-4bb1-b041-a2d85720be26\") "
Sep 16 19:04:08.346388 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.346361    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43702410-7574-4bb1-b041-a2d85720be26" (UID: "43702410-7574-4bb1-b041-a2d85720be26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:04:08.348255 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.348218    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr" (OuterVolumeSpecName: "kube-api-access-jrtnr") pod "43702410-7574-4bb1-b041-a2d85720be26" (UID: "43702410-7574-4bb1-b041-a2d85720be26"). InnerVolumeSpecName "kube-api-access-jrtnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:08.348551 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.348294    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43702410-7574-4bb1-b041-a2d85720be26" (UID: "43702410-7574-4bb1-b041-a2d85720be26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:04:08.348551 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.348355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43702410-7574-4bb1-b041-a2d85720be26" (UID: "43702410-7574-4bb1-b041-a2d85720be26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:04:08.447231 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.447205    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.447231 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.447231    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43702410-7574-4bb1-b041-a2d85720be26-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.447410 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.447243    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43702410-7574-4bb1-b041-a2d85720be26-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.447410 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.447252    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrtnr\" (UniqueName: \"kubernetes.io/projected/43702410-7574-4bb1-b041-a2d85720be26-kube-api-access-jrtnr\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.992578 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.992536    2576 generic.go:356] "Generic (PLEG): container finished" podID="43702410-7574-4bb1-b041-a2d85720be26" containerID="4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2" exitCode=0
Sep 16 19:04:08.992754 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.992586    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"
Sep 16 19:04:08.992754 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.992600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" event={"ID":"43702410-7574-4bb1-b041-a2d85720be26","Type":"ContainerDied","Data":"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"}
Sep 16 19:04:08.992754 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.992663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m" event={"ID":"43702410-7574-4bb1-b041-a2d85720be26","Type":"ContainerDied","Data":"3570fe634e5936a5ec903dee23bd24c13e42f3166afbeb031496ebd41e9570e4"}
Sep 16 19:04:08.992754 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:08.992691    2576 scope.go:117] "RemoveContainer" containerID="4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"
Sep 16 19:04:09.002597 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:09.002580    2576 scope.go:117] "RemoveContainer" containerID="4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"
Sep 16 19:04:09.002823 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:04:09.002805    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2\": container with ID starting with 4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2 not found: ID does not exist" containerID="4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"
Sep 16 19:04:09.002871 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:09.002833    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2"} err="failed to get container status \"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2\": rpc error: code = NotFound desc = could not find container \"4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2\": container with ID starting with 4129a9fad55ea5487f9df95ea4fe3e4fea3a8c21507e86583260738a1400e2f2 not found: ID does not exist"
Sep 16 19:04:09.018085 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:09.018062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:04:09.023265 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:09.023245    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a532e-58fcdfcf58-svf9m"]
Sep 16 19:04:11.010883 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:04:11.010842    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43702410-7574-4bb1-b041-a2d85720be26" path="/var/lib/kubelet/pods/43702410-7574-4bb1-b041-a2d85720be26/volumes"
Sep 16 19:05:10.276275 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.276236    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:05:10.276741 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.276449    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" podUID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" containerName="maas-api" containerID="cri-o://9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7" gracePeriod=30
Sep 16 19:05:10.537641 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.537576    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:05:10.578835 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.578802    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm8zj\" (UniqueName: \"kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj\") pod \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") "
Sep 16 19:05:10.579038 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.578898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs\") pod \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") "
Sep 16 19:05:10.579038 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.578945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca\") pod \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") "
Sep 16 19:05:10.579038 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.578974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") pod \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\" (UID: \"0a45707c-7cc5-4d04-9f94-b8f5247aac24\") "
Sep 16 19:05:10.579366 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.579340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a45707c-7cc5-4d04-9f94-b8f5247aac24" (UID: "0a45707c-7cc5-4d04-9f94-b8f5247aac24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:10.581460 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.581429    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a45707c-7cc5-4d04-9f94-b8f5247aac24" (UID: "0a45707c-7cc5-4d04-9f94-b8f5247aac24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:10.581609 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.581582    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj" (OuterVolumeSpecName: "kube-api-access-jm8zj") pod "0a45707c-7cc5-4d04-9f94-b8f5247aac24" (UID: "0a45707c-7cc5-4d04-9f94-b8f5247aac24"). InnerVolumeSpecName "kube-api-access-jm8zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:10.581718 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.581700    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0a45707c-7cc5-4d04-9f94-b8f5247aac24" (UID: "0a45707c-7cc5-4d04-9f94-b8f5247aac24"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:10.680171 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.680137    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:10.680171 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.680167    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm8zj\" (UniqueName: \"kubernetes.io/projected/0a45707c-7cc5-4d04-9f94-b8f5247aac24-kube-api-access-jm8zj\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:10.680171 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.680177    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a45707c-7cc5-4d04-9f94-b8f5247aac24-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:10.680410 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:10.680185    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a45707c-7cc5-4d04-9f94-b8f5247aac24-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:11.356225 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.356129    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" containerID="9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7" exitCode=0
Sep 16 19:05:11.356225 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.356195    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"
Sep 16 19:05:11.356743 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.356221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" event={"ID":"0a45707c-7cc5-4d04-9f94-b8f5247aac24","Type":"ContainerDied","Data":"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"}
Sep 16 19:05:11.356743 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.356278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn" event={"ID":"0a45707c-7cc5-4d04-9f94-b8f5247aac24","Type":"ContainerDied","Data":"56bf1f821a110b7b48c8cee71afe3e7f178a22e6aad05e6793fee32a9484a767"}
Sep 16 19:05:11.356743 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.356300    2576 scope.go:117] "RemoveContainer" containerID="9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"
Sep 16 19:05:11.365890 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.365868    2576 scope.go:117] "RemoveContainer" containerID="9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"
Sep 16 19:05:11.366184 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:05:11.366163    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7\": container with ID starting with 9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7 not found: ID does not exist" containerID="9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"
Sep 16 19:05:11.366225 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.366193    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7"} err="failed to get container status \"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7\": rpc error: code = NotFound desc = could not find container \"9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7\": container with ID starting with 9316f3ef74578bd922863e6ffb7f4fa176a035afbfac80d52e118266bf317ba7 not found: ID does not exist"
Sep 16 19:05:11.380834 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.380813    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:05:11.385678 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:11.385655    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a0f152-77f8d4c9d-zd8pn"]
Sep 16 19:05:13.010784 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:13.010741    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" path="/var/lib/kubelet/pods/0a45707c-7cc5-4d04-9f94-b8f5247aac24/volumes"
Sep 16 19:05:32.898617 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:32.898531    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 19:05:32.899455 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:32.899421    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" podUID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" containerName="maas-api" containerID="cri-o://85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec" gracePeriod=30
Sep 16 19:05:33.042663 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.042629    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 19:05:33.042849 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.042829    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" podUID="6be3c742-3497-4926-9328-750a84ea7f5f" containerName="payload-pre-processing" containerID="cri-o://1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03" gracePeriod=30
Sep 16 19:05:33.169039 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.169013    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 19:05:33.275691 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.275658    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca\") pod \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") "
Sep 16 19:05:33.275864 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.275703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs\") pod \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") "
Sep 16 19:05:33.275864 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.275737    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") pod \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") "
Sep 16 19:05:33.275864 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.275758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhqkf\" (UniqueName: \"kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf\") pod \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\" (UID: \"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c\") "
Sep 16 19:05:33.276141 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.276114    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" (UID: "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:33.278093 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.278061    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" (UID: "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:33.278208 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.278090    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" (UID: "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:33.278208 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.278109    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf" (OuterVolumeSpecName: "kube-api-access-xhqkf") pod "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" (UID: "ee472f3a-f0dd-4e62-9b80-02f1cbddd67c"). InnerVolumeSpecName "kube-api-access-xhqkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:33.295767 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.295746    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 19:05:33.376731 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.376698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd9bn\" (UniqueName: \"kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn\") pod \"6be3c742-3497-4926-9328-750a84ea7f5f\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") "
Sep 16 19:05:33.376884 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.376775    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume\") pod \"6be3c742-3497-4926-9328-750a84ea7f5f\" (UID: \"6be3c742-3497-4926-9328-750a84ea7f5f\") "
Sep 16 19:05:33.377158 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.377137    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6be3c742-3497-4926-9328-750a84ea7f5f" (UID: "6be3c742-3497-4926-9328-750a84ea7f5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:33.377239 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.377223    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.377278 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.377246    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.377278 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.377260    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.377278 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.377274    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhqkf\" (UniqueName: \"kubernetes.io/projected/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c-kube-api-access-xhqkf\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.379009 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.378990    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn" (OuterVolumeSpecName: "kube-api-access-qd9bn") pod "6be3c742-3497-4926-9328-750a84ea7f5f" (UID: "6be3c742-3497-4926-9328-750a84ea7f5f"). InnerVolumeSpecName "kube-api-access-qd9bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:33.477748 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.477720    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd9bn\" (UniqueName: \"kubernetes.io/projected/6be3c742-3497-4926-9328-750a84ea7f5f-kube-api-access-qd9bn\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.477748 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.477744    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be3c742-3497-4926-9328-750a84ea7f5f-plugins-config-volume\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.480340 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.480312    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" containerID="85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec" exitCode=0
Sep 16 19:05:33.480434 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.480355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"
Sep 16 19:05:33.480434 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.480394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" event={"ID":"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c","Type":"ContainerDied","Data":"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"}
Sep 16 19:05:33.480515 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.480453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p" event={"ID":"ee472f3a-f0dd-4e62-9b80-02f1cbddd67c","Type":"ContainerDied","Data":"2eabf2c06288b4fd1ed9c553e56d51b7459b605bdb2a7cc77f01878e805f170b"}
Sep 16 19:05:33.480515 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.480475    2576 scope.go:117] "RemoveContainer" containerID="85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"
Sep 16 19:05:33.481887 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.481859    2576 generic.go:356] "Generic (PLEG): container finished" podID="6be3c742-3497-4926-9328-750a84ea7f5f" containerID="1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03" exitCode=0
Sep 16 19:05:33.482020 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.481896    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"
Sep 16 19:05:33.482020 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.481892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" event={"ID":"6be3c742-3497-4926-9328-750a84ea7f5f","Type":"ContainerDied","Data":"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"}
Sep 16 19:05:33.482020 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.481961    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6" event={"ID":"6be3c742-3497-4926-9328-750a84ea7f5f","Type":"ContainerDied","Data":"df303885e3c4ad0d7bccab711c1e3bd9d8bb380d1b6d60941cf74aa8eefdcb9d"}
Sep 16 19:05:33.490183 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.490169    2576 scope.go:117] "RemoveContainer" containerID="85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"
Sep 16 19:05:33.490476 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:05:33.490457    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec\": container with ID starting with 85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec not found: ID does not exist" containerID="85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"
Sep 16 19:05:33.490557 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.490483    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec"} err="failed to get container status \"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec\": rpc error: code = NotFound desc = could not find container \"85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec\": container with ID starting with 85fb6c5be9f649b369ceea4f23185cf39608ccf85810348a9def79dbb155a3ec not found: ID does not exist"
Sep 16 19:05:33.490557 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.490501    2576 scope.go:117] "RemoveContainer" containerID="1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"
Sep 16 19:05:33.500159 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.500143    2576 scope.go:117] "RemoveContainer" containerID="1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"
Sep 16 19:05:33.500423 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:05:33.500404    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03\": container with ID starting with 1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03 not found: ID does not exist" containerID="1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"
Sep 16 19:05:33.500488 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.500428    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03"} err="failed to get container status \"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03\": rpc error: code = NotFound desc = could not find container \"1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03\": container with ID starting with 1960175a18d6d62801783aaadd7e36ec6b7c6fdd6e22367f01576374eef27a03 not found: ID does not exist"
Sep 16 19:05:33.527796 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.527771    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 19:05:33.533309 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.533286    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-f9c9b8-6c86fb6b54-jq72p"]
Sep 16 19:05:33.559791 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.559771    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 19:05:33.567087 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:33.567066    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f9c9b8-5b959ffcd6-gtjg6"]
Sep 16 19:05:35.011530 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:35.011496    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be3c742-3497-4926-9328-750a84ea7f5f" path="/var/lib/kubelet/pods/6be3c742-3497-4926-9328-750a84ea7f5f/volumes"
Sep 16 19:05:35.011900 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:35.011872    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" path="/var/lib/kubelet/pods/ee472f3a-f0dd-4e62-9b80-02f1cbddd67c/volumes"
Sep 16 19:05:55.026115 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.026080    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 19:05:55.026528 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.026297    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" podUID="41e56038-727a-42b4-a30a-05390088ccbe" containerName="maas-api" containerID="cri-o://f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060" gracePeriod=30
Sep 16 19:05:55.284618 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.284564    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 19:05:55.368576 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.368541    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls\") pod \"41e56038-727a-42b4-a30a-05390088ccbe\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") "
Sep 16 19:05:55.368576 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.368579    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca\") pod \"41e56038-727a-42b4-a30a-05390088ccbe\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") "
Sep 16 19:05:55.368800 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.368650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mdd2\" (UniqueName: \"kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2\") pod \"41e56038-727a-42b4-a30a-05390088ccbe\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") "
Sep 16 19:05:55.368800 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.368735    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs\") pod \"41e56038-727a-42b4-a30a-05390088ccbe\" (UID: \"41e56038-727a-42b4-a30a-05390088ccbe\") "
Sep 16 19:05:55.369072 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.369040    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41e56038-727a-42b4-a30a-05390088ccbe" (UID: "41e56038-727a-42b4-a30a-05390088ccbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:55.370993 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.370921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "41e56038-727a-42b4-a30a-05390088ccbe" (UID: "41e56038-727a-42b4-a30a-05390088ccbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:55.371111 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.370974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2" (OuterVolumeSpecName: "kube-api-access-8mdd2") pod "41e56038-727a-42b4-a30a-05390088ccbe" (UID: "41e56038-727a-42b4-a30a-05390088ccbe"). InnerVolumeSpecName "kube-api-access-8mdd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:55.371655 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.371629    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41e56038-727a-42b4-a30a-05390088ccbe" (UID: "41e56038-727a-42b4-a30a-05390088ccbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:55.470302 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.470265    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mdd2\" (UniqueName: \"kubernetes.io/projected/41e56038-727a-42b4-a30a-05390088ccbe-kube-api-access-8mdd2\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:55.470302 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.470302    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:55.470525 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.470315    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e56038-727a-42b4-a30a-05390088ccbe-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:55.470525 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.470327    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e56038-727a-42b4-a30a-05390088ccbe-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:05:55.615000 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.614895    2576 generic.go:356] "Generic (PLEG): container finished" podID="41e56038-727a-42b4-a30a-05390088ccbe" containerID="f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060" exitCode=0
Sep 16 19:05:55.615000 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.614965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" event={"ID":"41e56038-727a-42b4-a30a-05390088ccbe","Type":"ContainerDied","Data":"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"}
Sep 16 19:05:55.615000 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.614980    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"
Sep 16 19:05:55.615243 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.615019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw" event={"ID":"41e56038-727a-42b4-a30a-05390088ccbe","Type":"ContainerDied","Data":"703a4518a9b2dda816ce916f09a94cf3e37fb88afcc09fc7d0035bf8ae01d7fc"}
Sep 16 19:05:55.615243 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.615050    2576 scope.go:117] "RemoveContainer" containerID="f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"
Sep 16 19:05:55.626146 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.626108    2576 scope.go:117] "RemoveContainer" containerID="f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"
Sep 16 19:05:55.626450 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:05:55.626430    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060\": container with ID starting with f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060 not found: ID does not exist" containerID="f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"
Sep 16 19:05:55.626534 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.626459    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060"} err="failed to get container status \"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060\": rpc error: code = NotFound desc = could not find container \"f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060\": container with ID starting with f31fcf66e95448032885942c271bf5b47f98e9458249d5214a5f21bb35ab1060 not found: ID does not exist"
Sep 16 19:05:55.642156 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.642131    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 19:05:55.649702 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:55.649674    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e65cc4-5898746756-zcwnw"]
Sep 16 19:05:57.011103 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:05:57.011066    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e56038-727a-42b4-a30a-05390088ccbe" path="/var/lib/kubelet/pods/41e56038-727a-42b4-a30a-05390088ccbe/volumes"
Sep 16 19:06:10.671328 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.671292    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 19:06:10.671846 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.671497    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerName="manager" containerID="cri-o://151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff" gracePeriod=30
Sep 16 19:06:10.917356 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.917328    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 19:06:10.994808 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.994731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs\") pod \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") "
Sep 16 19:06:10.994808 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.994791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert\") pod \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") "
Sep 16 19:06:10.995032 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.994811    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl85b\" (UniqueName: \"kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b\") pod \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\" (UID: \"2ca49a7f-8c40-4076-bd42-7f3a21892c9a\") "
Sep 16 19:06:10.997009 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.996975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2ca49a7f-8c40-4076-bd42-7f3a21892c9a" (UID: "2ca49a7f-8c40-4076-bd42-7f3a21892c9a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:06:10.997554 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.997531    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2ca49a7f-8c40-4076-bd42-7f3a21892c9a" (UID: "2ca49a7f-8c40-4076-bd42-7f3a21892c9a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:06:10.997628 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:10.997549    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b" (OuterVolumeSpecName: "kube-api-access-dl85b") pod "2ca49a7f-8c40-4076-bd42-7f3a21892c9a" (UID: "2ca49a7f-8c40-4076-bd42-7f3a21892c9a"). InnerVolumeSpecName "kube-api-access-dl85b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:11.095783 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.095751    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:06:11.095783 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.095786    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-webhook-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:06:11.096016 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.095800    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl85b\" (UniqueName: \"kubernetes.io/projected/2ca49a7f-8c40-4076-bd42-7f3a21892c9a-kube-api-access-dl85b\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:06:11.712531 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.712495    2576 generic.go:356] "Generic (PLEG): container finished" podID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerID="151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff" exitCode=0
Sep 16 19:06:11.713019 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.712544    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-lvkn2"
Sep 16 19:06:11.713019 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.712558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" event={"ID":"2ca49a7f-8c40-4076-bd42-7f3a21892c9a","Type":"ContainerDied","Data":"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"}
Sep 16 19:06:11.713019 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.712603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" event={"ID":"2ca49a7f-8c40-4076-bd42-7f3a21892c9a","Type":"ContainerDied","Data":"bc241650b7a4c58a7dffaea6467b66b4514b1d501d018321480ac366721e9901"}
Sep 16 19:06:11.713019 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.712624    2576 scope.go:117] "RemoveContainer" containerID="151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"
Sep 16 19:06:11.723822 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.723806    2576 scope.go:117] "RemoveContainer" containerID="151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"
Sep 16 19:06:11.724138 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:06:11.724109    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff\": container with ID starting with 151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff not found: ID does not exist" containerID="151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"
Sep 16 19:06:11.724221 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.724145    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff"} err="failed to get container status \"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff\": rpc error: code = NotFound desc = could not find container \"151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff\": container with ID starting with 151afe8865c710669f3ff198dac5024eda0a05d811fe28e90a7f2ea9174e45ff not found: ID does not exist"
Sep 16 19:06:11.734977 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.734943    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 19:06:11.740065 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.740033    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-lvkn2"]
Sep 16 19:06:11.871398 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:11.871367    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-8d84d5c46-lvkn2" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.45:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 19:06:13.010885 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:06:13.010854    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" path="/var/lib/kubelet/pods/2ca49a7f-8c40-4076-bd42-7f3a21892c9a/volumes"
Sep 16 19:07:12.617438 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.617398    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.617969    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerName="manager"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.617986    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerName="manager"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618000    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43702410-7574-4bb1-b041-a2d85720be26" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618007    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="43702410-7574-4bb1-b041-a2d85720be26" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618012    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618018    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618034    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41e56038-727a-42b4-a30a-05390088ccbe" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618039    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41e56038-727a-42b4-a30a-05390088ccbe" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618049    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be3c742-3497-4926-9328-750a84ea7f5f" containerName="payload-pre-processing"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618068    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be3c742-3497-4926-9328-750a84ea7f5f" containerName="payload-pre-processing"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618075    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1663fef6-0100-4e77-a094-c722bb0a4b2f" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618080    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1663fef6-0100-4e77-a094-c722bb0a4b2f" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618091    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618096    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618155    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41e56038-727a-42b4-a30a-05390088ccbe" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618164    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be3c742-3497-4926-9328-750a84ea7f5f" containerName="payload-pre-processing"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618171    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca49a7f-8c40-4076-bd42-7f3a21892c9a" containerName="manager"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618177    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="43702410-7574-4bb1-b041-a2d85720be26" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618182    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1663fef6-0100-4e77-a094-c722bb0a4b2f" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618189    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a45707c-7cc5-4d04-9f94-b8f5247aac24" containerName="maas-api"
Sep 16 19:07:12.619956 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.618196    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee472f3a-f0dd-4e62-9b80-02f1cbddd67c" containerName="maas-api"
Sep 16 19:07:12.621183 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.621164    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.624079 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.624052    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 19:07:12.624195 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.624105    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cgvng\""
Sep 16 19:07:12.624888 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.624856    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 19:07:12.625025 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.624906    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 19:07:12.637215 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.637191    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:12.700705 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.700680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.700830 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.700713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npbdg\" (UniqueName: \"kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.700830 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.700806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.801715 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.801684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.801816 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.801746    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.801816 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.801772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npbdg\" (UniqueName: \"kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.804340 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.804320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.804433 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.804350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.813089 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.813066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npbdg\" (UniqueName: \"kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg\") pod \"maas-controller-8d84d5c46-qzsxw\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") " pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:12.931616 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:12.931596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:13.214185 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:13.214162    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:13.215793 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:07:13.215762    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f638daf_ce8a_409b_a3b3_d8baa8a5f06f.slice/crio-506379beacbd96a36e1b4ec06133ea2ee2dfc94c412605638e07e34370ce807b WatchSource:0}: Error finding container 506379beacbd96a36e1b4ec06133ea2ee2dfc94c412605638e07e34370ce807b: Status 404 returned error can't find the container with id 506379beacbd96a36e1b4ec06133ea2ee2dfc94c412605638e07e34370ce807b
Sep 16 19:07:13.217116 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:13.217097    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:07:14.065440 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:14.065409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" event={"ID":"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f","Type":"ContainerStarted","Data":"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"}
Sep 16 19:07:14.065923 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:14.065458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" event={"ID":"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f","Type":"ContainerStarted","Data":"506379beacbd96a36e1b4ec06133ea2ee2dfc94c412605638e07e34370ce807b"}
Sep 16 19:07:14.065923 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:14.065489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:14.101844 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:14.101789    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" podStartSLOduration=1.409686964 podStartE2EDuration="2.101771301s" totalImagesPullingTime="692.084337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:07:14.099537691 +0000 UTC m=+2361.520024562" watchObservedRunningTime="2026-09-16 19:07:14.101771301 +0000 UTC m=+2361.522258167"
Sep 16 19:07:25.073461 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:25.073425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:34.495418 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.495376    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:34.495883 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.495631    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" podUID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" containerName="manager" containerID="cri-o://45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113" gracePeriod=30
Sep 16 19:07:34.744649 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.744623    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:34.890909 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.890836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert\") pod \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") "
Sep 16 19:07:34.891093 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.890947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs\") pod \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") "
Sep 16 19:07:34.891093 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.891027    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npbdg\" (UniqueName: \"kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg\") pod \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\" (UID: \"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f\") "
Sep 16 19:07:34.893059 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.893032    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" (UID: "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:07:34.893175 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.893103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" (UID: "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:07:34.893175 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.893107    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg" (OuterVolumeSpecName: "kube-api-access-npbdg") pod "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" (UID: "3f638daf-ce8a-409b-a3b3-d8baa8a5f06f"). InnerVolumeSpecName "kube-api-access-npbdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:07:34.999277 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.993370    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-webhook-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:07:34.999277 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.993403    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:07:34.999277 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:34.993420    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npbdg\" (UniqueName: \"kubernetes.io/projected/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f-kube-api-access-npbdg\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:07:35.183783 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.183752    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" containerID="45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113" exitCode=0
Sep 16 19:07:35.183906 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.183798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" event={"ID":"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f","Type":"ContainerDied","Data":"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"}
Sep 16 19:07:35.183906 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.183851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-qzsxw" event={"ID":"3f638daf-ce8a-409b-a3b3-d8baa8a5f06f","Type":"ContainerDied","Data":"506379beacbd96a36e1b4ec06133ea2ee2dfc94c412605638e07e34370ce807b"}
Sep 16 19:07:35.183906 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.183874    2576 scope.go:117] "RemoveContainer" containerID="45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"
Sep 16 19:07:35.183906 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.183808    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-qzsxw"
Sep 16 19:07:35.196799 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.196781    2576 scope.go:117] "RemoveContainer" containerID="45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"
Sep 16 19:07:35.197181 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:07:35.197159    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113\": container with ID starting with 45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113 not found: ID does not exist" containerID="45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"
Sep 16 19:07:35.197287 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.197189    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113"} err="failed to get container status \"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113\": rpc error: code = NotFound desc = could not find container \"45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113\": container with ID starting with 45655f58103cbccdda9b6007b6a993dbabe4d8c712b06e1386e6003be4b1c113 not found: ID does not exist"
Sep 16 19:07:35.207886 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.207857    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:35.214855 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.214835    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-qzsxw"]
Sep 16 19:07:35.996633 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.996603    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8d84d5c46-ngwsl"]
Sep 16 19:07:35.997060 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.997043    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" containerName="manager"
Sep 16 19:07:35.997113 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.997062    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" containerName="manager"
Sep 16 19:07:35.997175 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:35.997165    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" containerName="manager"
Sep 16 19:07:36.001592 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.001572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.004182 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.004164    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 19:07:36.005396 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.005373    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 19:07:36.005543 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.005409    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 19:07:36.005543 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.005417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cgvng\""
Sep 16 19:07:36.014276 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.014254    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-ngwsl"]
Sep 16 19:07:36.101692 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.101664    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-webhook-cert\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.101819 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.101714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-metrics-certs\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.101819 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.101801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvd56\" (UniqueName: \"kubernetes.io/projected/99e10a4d-a361-4d3c-bcb8-ec493e470de3-kube-api-access-rvd56\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.203051 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.203026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-webhook-cert\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.203163 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.203067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-metrics-certs\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.203163 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.203144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvd56\" (UniqueName: \"kubernetes.io/projected/99e10a4d-a361-4d3c-bcb8-ec493e470de3-kube-api-access-rvd56\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.205728 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.205703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-webhook-cert\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.205813 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.205757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e10a4d-a361-4d3c-bcb8-ec493e470de3-metrics-certs\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.219802 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.219776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvd56\" (UniqueName: \"kubernetes.io/projected/99e10a4d-a361-4d3c-bcb8-ec493e470de3-kube-api-access-rvd56\") pod \"maas-controller-8d84d5c46-ngwsl\" (UID: \"99e10a4d-a361-4d3c-bcb8-ec493e470de3\") " pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.313893 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.313834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:36.590735 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:36.590706    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d84d5c46-ngwsl"]
Sep 16 19:07:36.591148 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:07:36.591125    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e10a4d_a361_4d3c_bcb8_ec493e470de3.slice/crio-1a049b18aafed4302e8d7d57be120475089bc25e1cf904412ab58465cc4f2ec3 WatchSource:0}: Error finding container 1a049b18aafed4302e8d7d57be120475089bc25e1cf904412ab58465cc4f2ec3: Status 404 returned error can't find the container with id 1a049b18aafed4302e8d7d57be120475089bc25e1cf904412ab58465cc4f2ec3
Sep 16 19:07:37.011416 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:37.011385    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f638daf-ce8a-409b-a3b3-d8baa8a5f06f" path="/var/lib/kubelet/pods/3f638daf-ce8a-409b-a3b3-d8baa8a5f06f/volumes"
Sep 16 19:07:37.197596 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:37.197566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-ngwsl" event={"ID":"99e10a4d-a361-4d3c-bcb8-ec493e470de3","Type":"ContainerStarted","Data":"8388e7e9635cdb4c72c9c9bcc7c1da27d155dfe8b620bb7a9d157a40b7d1fb9c"}
Sep 16 19:07:37.197730 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:37.197618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d84d5c46-ngwsl" event={"ID":"99e10a4d-a361-4d3c-bcb8-ec493e470de3","Type":"ContainerStarted","Data":"1a049b18aafed4302e8d7d57be120475089bc25e1cf904412ab58465cc4f2ec3"}
Sep 16 19:07:37.197730 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:37.197683    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:07:37.219860 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:37.219816    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8d84d5c46-ngwsl" podStartSLOduration=1.75704357 podStartE2EDuration="2.219799296s" totalImagesPullingTime="462.755726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:07:37.218431828 +0000 UTC m=+2384.638918688" watchObservedRunningTime="2026-09-16 19:07:37.219799296 +0000 UTC m=+2384.640286157"
Sep 16 19:07:48.206041 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:07:48.206005    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8d84d5c46-ngwsl"
Sep 16 19:12:08.784889 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.784855    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:08.789285 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.789268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:08.794681 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.794646    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lpqld\""
Sep 16 19:12:08.795011 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.794885    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 19:12:08.795011 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.794975    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 19:12:08.795195 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.795166    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 19:12:08.795493 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.795471    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"]
Sep 16 19:12:08.797029 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.796770    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 19:12:08.799141 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.799121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:08.832089 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.832068    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"]
Sep 16 19:12:08.867822 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.867795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:08.913525 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcr4h\" (UniqueName: \"kubernetes.io/projected/1f933fab-4232-45d7-9199-5205dc1b47a5-kube-api-access-gcr4h\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:08.913643 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-maas-api-tls\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:08.913643 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:08.913643 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-metrics-certs\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:08.913643 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw9nf\" (UniqueName: \"kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:08.913643 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:08.913817 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:08.913817 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:08.913729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f933fab-4232-45d7-9199-5205dc1b47a5-openshift-service-ca\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.017279 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f933fab-4232-45d7-9199-5205dc1b47a5-openshift-service-ca\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.017471 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcr4h\" (UniqueName: \"kubernetes.io/projected/1f933fab-4232-45d7-9199-5205dc1b47a5-kube-api-access-gcr4h\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.017536 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-maas-api-tls\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.017536 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.017642 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-metrics-certs\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.017642 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw9nf\" (UniqueName: \"kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.017642 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.017642 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.017609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.018210 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.018184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f933fab-4232-45d7-9199-5205dc1b47a5-openshift-service-ca\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.018453 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.018384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.020484 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.020457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.020728 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.020703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.020890 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.020872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-maas-api-tls\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.021025 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.020951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f933fab-4232-45d7-9199-5205dc1b47a5-metrics-certs\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.034127 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.034106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcr4h\" (UniqueName: \"kubernetes.io/projected/1f933fab-4232-45d7-9199-5205dc1b47a5-kube-api-access-gcr4h\") pod \"maas-api-6bdc786cdd-tc229\" (UID: \"1f933fab-4232-45d7-9199-5205dc1b47a5\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.036031 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.035979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw9nf\" (UniqueName: \"kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf\") pod \"maas-api-67757c4776-bgmx4\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.100488 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.100463    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:09.109411 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.109392    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:09.412858 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.412635    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:09.416187 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:12:09.416158    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d1ebf5_b12c_4555_9939_aa041e478d01.slice/crio-106a749d4966657319d2bfc03ec7b79454510cdbe7a7e49b042f7b4add1bed11 WatchSource:0}: Error finding container 106a749d4966657319d2bfc03ec7b79454510cdbe7a7e49b042f7b4add1bed11: Status 404 returned error can't find the container with id 106a749d4966657319d2bfc03ec7b79454510cdbe7a7e49b042f7b4add1bed11
Sep 16 19:12:09.416843 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:12:09.416810    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f933fab_4232_45d7_9199_5205dc1b47a5.slice/crio-180930ba97d9016896cc32b7f29f51de1f3068251e7ddabfb89ac569cfd92c72 WatchSource:0}: Error finding container 180930ba97d9016896cc32b7f29f51de1f3068251e7ddabfb89ac569cfd92c72: Status 404 returned error can't find the container with id 180930ba97d9016896cc32b7f29f51de1f3068251e7ddabfb89ac569cfd92c72
Sep 16 19:12:09.416991 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.416860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"]
Sep 16 19:12:09.747148 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.747107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229" event={"ID":"1f933fab-4232-45d7-9199-5205dc1b47a5","Type":"ContainerStarted","Data":"180930ba97d9016896cc32b7f29f51de1f3068251e7ddabfb89ac569cfd92c72"}
Sep 16 19:12:09.748318 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:09.748300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" event={"ID":"22d1ebf5-b12c-4555-9939-aa041e478d01","Type":"ContainerStarted","Data":"106a749d4966657319d2bfc03ec7b79454510cdbe7a7e49b042f7b4add1bed11"}
Sep 16 19:12:10.755138 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.755106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229" event={"ID":"1f933fab-4232-45d7-9199-5205dc1b47a5","Type":"ContainerStarted","Data":"46181b0e94854487fce42312cd3a92467242c4aa0f36af5460bb0f187da2ad75"}
Sep 16 19:12:10.755598 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.755297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:10.756683 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.756663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" event={"ID":"22d1ebf5-b12c-4555-9939-aa041e478d01","Type":"ContainerStarted","Data":"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"}
Sep 16 19:12:10.756735 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.756710    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:10.807510 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.807456    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229" podStartSLOduration=1.998565999 podStartE2EDuration="2.807439613s" totalImagesPullingTime="808.873614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:10.806959032 +0000 UTC m=+2658.227445885" watchObservedRunningTime="2026-09-16 19:12:10.807439613 +0000 UTC m=+2658.227926473"
Sep 16 19:12:10.844343 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:10.844294    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" podStartSLOduration=2.424160004 podStartE2EDuration="2.844276459s" totalImagesPullingTime="420.116455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:10.843289454 +0000 UTC m=+2658.263776313" watchObservedRunningTime="2026-09-16 19:12:10.844276459 +0000 UTC m=+2658.264763319"
Sep 16 19:12:16.767717 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:16.767688    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:16.770215 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:16.768122    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-tc229"
Sep 16 19:12:17.128137 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.128059    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:17.128282 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.128227    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" podUID="22d1ebf5-b12c-4555-9939-aa041e478d01" containerName="maas-api" containerID="cri-o://47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf" gracePeriod=30
Sep 16 19:12:17.367759 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.367738    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:17.394248 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394179    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca\") pod \"22d1ebf5-b12c-4555-9939-aa041e478d01\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") "
Sep 16 19:12:17.394361 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394292    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls\") pod \"22d1ebf5-b12c-4555-9939-aa041e478d01\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") "
Sep 16 19:12:17.394427 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394362    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw9nf\" (UniqueName: \"kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf\") pod \"22d1ebf5-b12c-4555-9939-aa041e478d01\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") "
Sep 16 19:12:17.394427 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394402    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs\") pod \"22d1ebf5-b12c-4555-9939-aa041e478d01\" (UID: \"22d1ebf5-b12c-4555-9939-aa041e478d01\") "
Sep 16 19:12:17.394665 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394597    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22d1ebf5-b12c-4555-9939-aa041e478d01" (UID: "22d1ebf5-b12c-4555-9939-aa041e478d01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:17.394778 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.394713    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d1ebf5-b12c-4555-9939-aa041e478d01-openshift-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:12:17.396718 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.396685    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "22d1ebf5-b12c-4555-9939-aa041e478d01" (UID: "22d1ebf5-b12c-4555-9939-aa041e478d01"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:17.396831 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.396730    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22d1ebf5-b12c-4555-9939-aa041e478d01" (UID: "22d1ebf5-b12c-4555-9939-aa041e478d01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:17.396831 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.396751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf" (OuterVolumeSpecName: "kube-api-access-kw9nf") pod "22d1ebf5-b12c-4555-9939-aa041e478d01" (UID: "22d1ebf5-b12c-4555-9939-aa041e478d01"). InnerVolumeSpecName "kube-api-access-kw9nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:17.495329 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.495302    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-maas-api-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:12:17.495329 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.495327    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw9nf\" (UniqueName: \"kubernetes.io/projected/22d1ebf5-b12c-4555-9939-aa041e478d01-kube-api-access-kw9nf\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:12:17.495490 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.495336    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d1ebf5-b12c-4555-9939-aa041e478d01-metrics-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Sep 16 19:12:17.804648 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.804615    2576 generic.go:356] "Generic (PLEG): container finished" podID="22d1ebf5-b12c-4555-9939-aa041e478d01" containerID="47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf" exitCode=0
Sep 16 19:12:17.805128 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.804670    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"
Sep 16 19:12:17.805128 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.804695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" event={"ID":"22d1ebf5-b12c-4555-9939-aa041e478d01","Type":"ContainerDied","Data":"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"}
Sep 16 19:12:17.805128 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.804747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-bgmx4" event={"ID":"22d1ebf5-b12c-4555-9939-aa041e478d01","Type":"ContainerDied","Data":"106a749d4966657319d2bfc03ec7b79454510cdbe7a7e49b042f7b4add1bed11"}
Sep 16 19:12:17.805128 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.804768    2576 scope.go:117] "RemoveContainer" containerID="47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"
Sep 16 19:12:17.814788 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.814761    2576 scope.go:117] "RemoveContainer" containerID="47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"
Sep 16 19:12:17.815168 ip-10-0-131-165 kubenswrapper[2576]: E0916 19:12:17.815149    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf\": container with ID starting with 47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf not found: ID does not exist" containerID="47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"
Sep 16 19:12:17.815249 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.815180    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf"} err="failed to get container status \"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf\": rpc error: code = NotFound desc = could not find container \"47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf\": container with ID starting with 47218dcd3c07b65151556d564c18b2a9f6105ffe6a501fdb8303b7caf012adbf not found: ID does not exist"
Sep 16 19:12:17.860839 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.860816    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:17.908152 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:17.908123    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-bgmx4"]
Sep 16 19:12:19.011526 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:19.011492    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d1ebf5-b12c-4555-9939-aa041e478d01" path="/var/lib/kubelet/pods/22d1ebf5-b12c-4555-9939-aa041e478d01/volumes"
Sep 16 19:12:46.387523 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:46.387433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8d84d5c46-ngwsl_99e10a4d-a361-4d3c-bcb8-ec493e470de3/manager/0.log"
Sep 16 19:12:46.513449 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:46.513417    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9bvb7_6acde495-51ab-4b52-8a17-3ecd6e634897/server/0.log"
Sep 16 19:12:46.642605 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:46.642529    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mv5wh_51fa60a8-66ab-47f1-8b7e-ac798e054f4f/manager/0.log"
Sep 16 19:12:47.121615 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:47.121589    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-744584488c-zjsgc_59855f7f-7943-459d-affd-eb22be5ebc6c/copy-manifests/0.log"
Sep 16 19:12:47.143383 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:47.143353    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-744584488c-zjsgc_59855f7f-7943-459d-affd-eb22be5ebc6c/manager/0.log"
Sep 16 19:12:47.698003 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:47.697967    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bdc786cdd-tc229_1f933fab-4232-45d7-9199-5205dc1b47a5/maas-api/0.log"
Sep 16 19:12:50.443582 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:50.443549    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h9gc2_7ee0d139-65db-4125-9880-d74fb265c56e/discovery/0.log"
Sep 16 19:12:51.204826 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:51.204795    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:12:51.757945 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:12:51.757914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-62p84_af44e0d9-a5f6-44a8-930f-2d06417e52b6/main/0.log"
Sep 16 19:13:02.701920 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:02.701889    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5fns9_7c049995-694f-4ace-b1a7-6ea9f6d047a7/global-pull-secret-syncer/0.log"
Sep 16 19:13:03.707208 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:03.707178    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x8zxq_182dea7c-2698-468e-88bc-428ad9033d36/konnectivity-agent/0.log"
Sep 16 19:13:03.830139 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:03.830116    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-165.ec2.internal_16cc17fcb523a0845ceaeb734d781a6a/haproxy/0.log"
Sep 16 19:13:09.362315 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.362289    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/alertmanager/0.log"
Sep 16 19:13:09.476034 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.476009    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/config-reloader/0.log"
Sep 16 19:13:09.574522 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.574470    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/kube-rbac-proxy-web/0.log"
Sep 16 19:13:09.665739 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.665649    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/kube-rbac-proxy/0.log"
Sep 16 19:13:09.772270 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.772245    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/kube-rbac-proxy-metric/0.log"
Sep 16 19:13:09.871350 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.871321    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/prom-label-proxy/0.log"
Sep 16 19:13:09.949726 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:09.949701    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c8e6668a-5d23-4250-8147-49be791c6824/init-config-reloader/0.log"
Sep 16 19:13:10.068783 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.068750    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-mlb96_a5c79871-1520-4581-8a74-d4c1e6ca07d3/cluster-monitoring-operator/0.log"
Sep 16 19:13:10.150371 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.150340    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-vl425_a625b437-cd99-45ab-bc91-e76a67aa7528/kube-state-metrics/0.log"
Sep 16 19:13:10.286904 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.286795    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-vl425_a625b437-cd99-45ab-bc91-e76a67aa7528/kube-rbac-proxy-main/0.log"
Sep 16 19:13:10.355531 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.355473    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-vl425_a625b437-cd99-45ab-bc91-e76a67aa7528/kube-rbac-proxy-self/0.log"
Sep 16 19:13:10.519910 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.519880    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-k68qr_45fee104-8448-41a8-a471-7bd19f193c58/monitoring-plugin/0.log"
Sep 16 19:13:10.809335 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.809303    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w9hh8_6de8231e-5c24-47f2-8f6a-c93ffaf79435/node-exporter/0.log"
Sep 16 19:13:10.866783 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.866753    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w9hh8_6de8231e-5c24-47f2-8f6a-c93ffaf79435/kube-rbac-proxy/0.log"
Sep 16 19:13:10.952411 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:10.952387    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w9hh8_6de8231e-5c24-47f2-8f6a-c93ffaf79435/init-textfile/0.log"
Sep 16 19:13:11.321790 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.321762    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljz5b_83bcd5f1-d03e-4d66-bcc6-022302423c73/kube-rbac-proxy-main/0.log"
Sep 16 19:13:11.359316 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.359285    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"]
Sep 16 19:13:11.359696 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.359682    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d1ebf5-b12c-4555-9939-aa041e478d01" containerName="maas-api"
Sep 16 19:13:11.359745 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.359698    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d1ebf5-b12c-4555-9939-aa041e478d01" containerName="maas-api"
Sep 16 19:13:11.359781 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.359777    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d1ebf5-b12c-4555-9939-aa041e478d01" containerName="maas-api"
Sep 16 19:13:11.363148 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.363127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.375551 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.375535    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km8mc\"/\"kube-root-ca.crt\""
Sep 16 19:13:11.376287 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.376270    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km8mc\"/\"openshift-service-ca.crt\""
Sep 16 19:13:11.391023 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.391001    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-km8mc\"/\"default-dockercfg-5tbwq\""
Sep 16 19:13:11.429861 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.429826    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljz5b_83bcd5f1-d03e-4d66-bcc6-022302423c73/kube-rbac-proxy-self/0.log"
Sep 16 19:13:11.452925 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.452897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhg9p\" (UniqueName: \"kubernetes.io/projected/34198f7a-565e-4e98-93f1-81a8c351edc8-kube-api-access-fhg9p\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.453081 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.452953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-proc\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.453081 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.453001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-podres\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.453081 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.453074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-lib-modules\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.453214 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.453101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-sys\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.468405 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.468375    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"]
Sep 16 19:13:11.516257 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.516233    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-ljz5b_83bcd5f1-d03e-4d66-bcc6-022302423c73/openshift-state-metrics/0.log"
Sep 16 19:13:11.553785 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-lib-modules\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-sys\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhg9p\" (UniqueName: \"kubernetes.io/projected/34198f7a-565e-4e98-93f1-81a8c351edc8-kube-api-access-fhg9p\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-proc\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-podres\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-lib-modules\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.553975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-sys\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.554004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-proc\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.554205 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.554062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34198f7a-565e-4e98-93f1-81a8c351edc8-podres\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.566734 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.566706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhg9p\" (UniqueName: \"kubernetes.io/projected/34198f7a-565e-4e98-93f1-81a8c351edc8-kube-api-access-fhg9p\") pod \"perf-node-gather-daemonset-clx4n\" (UID: \"34198f7a-565e-4e98-93f1-81a8c351edc8\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.673046 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.673020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:11.955271 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:11.955244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"]
Sep 16 19:13:11.957624 ip-10-0-131-165 kubenswrapper[2576]: W0916 19:13:11.957600    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34198f7a_565e_4e98_93f1_81a8c351edc8.slice/crio-f363635aa1b19e32ba53c13f8686b5a963a4f6eb72e1d2bd94cb986d0eb867dd WatchSource:0}: Error finding container f363635aa1b19e32ba53c13f8686b5a963a4f6eb72e1d2bd94cb986d0eb867dd: Status 404 returned error can't find the container with id f363635aa1b19e32ba53c13f8686b5a963a4f6eb72e1d2bd94cb986d0eb867dd
Sep 16 19:13:12.115764 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.115732    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n" event={"ID":"34198f7a-565e-4e98-93f1-81a8c351edc8","Type":"ContainerStarted","Data":"f5c2c14f8b4a169820588a9e1c2d45b725f9358f77e00855e838d1c6acfc1ae8"}
Sep 16 19:13:12.115869 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.115791    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n" event={"ID":"34198f7a-565e-4e98-93f1-81a8c351edc8","Type":"ContainerStarted","Data":"f363635aa1b19e32ba53c13f8686b5a963a4f6eb72e1d2bd94cb986d0eb867dd"}
Sep 16 19:13:12.115869 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.115830    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:12.144356 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.144313    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n" podStartSLOduration=1.144297623 podStartE2EDuration="1.144297623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:13:12.142259935 +0000 UTC m=+2719.562746794" watchObservedRunningTime="2026-09-16 19:13:12.144297623 +0000 UTC m=+2719.564784483"
Sep 16 19:13:12.342562 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.342489    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c5b4957-np4rj_6f91dae3-62e0-4cb3-9019-ffe0ceb3675e/telemeter-client/0.log"
Sep 16 19:13:12.397800 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.397774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c5b4957-np4rj_6f91dae3-62e0-4cb3-9019-ffe0ceb3675e/reload/0.log"
Sep 16 19:13:12.477555 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.477530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c5b4957-np4rj_6f91dae3-62e0-4cb3-9019-ffe0ceb3675e/kube-rbac-proxy/0.log"
Sep 16 19:13:12.547353 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.547316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/thanos-query/0.log"
Sep 16 19:13:12.646377 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.646315    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/kube-rbac-proxy-web/0.log"
Sep 16 19:13:12.697709 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.697693    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/kube-rbac-proxy/0.log"
Sep 16 19:13:12.760829 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.760811    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/prom-label-proxy/0.log"
Sep 16 19:13:12.849209 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.849192    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/kube-rbac-proxy-rules/0.log"
Sep 16 19:13:12.937857 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:12.937841    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8976999f-7b98l_0482fc37-0610-4cba-9562-4fb6d8c356c1/kube-rbac-proxy-metrics/0.log"
Sep 16 19:13:15.679458 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:15.679432    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-796498f4c7-qpd8g_cd762a9f-a7ea-48d9-9645-42bce6b568db/console/0.log"
Sep 16 19:13:15.743630 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:15.743606    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-bdjqk_7be202e6-ef77-4898-9f28-8b5bc58d1e10/download-server/0.log"
Sep 16 19:13:18.132193 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:18.132166    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-clx4n"
Sep 16 19:13:18.332348 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:18.332321    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lrwf8_b69b3521-a79f-44b0-99d8-9dc6d045f3c4/dns/0.log"
Sep 16 19:13:18.365985 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:18.365955    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lrwf8_b69b3521-a79f-44b0-99d8-9dc6d045f3c4/kube-rbac-proxy/0.log"
Sep 16 19:13:18.496593 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:18.496569    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xxksn_0b06ca00-79eb-4e58-90bb-af7533456332/dns-node-resolver/0.log"
Sep 16 19:13:19.198228 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:19.198199    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d89597d46-nnpch_6279a734-dcbb-4640-9912-60eeddc3144f/registry/0.log"
Sep 16 19:13:19.331895 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:19.331867    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5g7ts_8eed79a5-6b78-4e0d-8516-cdea9b3f726d/node-ca/0.log"
Sep 16 19:13:21.590725 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:21.590699    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h9gc2_7ee0d139-65db-4125-9880-d74fb265c56e/discovery/0.log"
Sep 16 19:13:21.960538 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:21.960510    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:13:23.108063 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:23.107989    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h9gc2_7ee0d139-65db-4125-9880-d74fb265c56e/discovery/0.log"
Sep 16 19:13:23.268125 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:23.268093    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-864pq_1a8ec31e-9939-4934-87a8-5119122e5001/serve-healthcheck-canary/0.log"
Sep 16 19:13:23.356484 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:23.356455    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:13:24.091766 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:24.091723    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-czsgb_ef541df6-cc2e-4d36-b932-392a6411733b/insights-operator/0.log"
Sep 16 19:13:24.092062 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:24.092044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-czsgb_ef541df6-cc2e-4d36-b932-392a6411733b/insights-operator/1.log"
Sep 16 19:13:24.769287 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:24.769264    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqhhh_dd37ec71-986a-45dc-ae05-4bdd19df7929/kube-rbac-proxy/0.log"
Sep 16 19:13:24.868140 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:24.868116    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqhhh_dd37ec71-986a-45dc-ae05-4bdd19df7929/exporter/0.log"
Sep 16 19:13:24.976525 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:24.976498    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqhhh_dd37ec71-986a-45dc-ae05-4bdd19df7929/extractor/0.log"
Sep 16 19:13:25.841089 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:25.841060    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h9gc2_7ee0d139-65db-4125-9880-d74fb265c56e/discovery/0.log"
Sep 16 19:13:26.306130 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:26.306103    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8cjcd_d485e4b5-5877-44d5-9e35-767ee27bb96f/payload-processing/0.log"
Sep 16 19:13:29.673281 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:29.673253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8d84d5c46-ngwsl_99e10a4d-a361-4d3c-bcb8-ec493e470de3/manager/0.log"
Sep 16 19:13:29.761885 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:29.761853    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9bvb7_6acde495-51ab-4b52-8a17-3ecd6e634897/server/0.log"
Sep 16 19:13:29.898674 ip-10-0-131-165 kubenswrapper[2576]: I0916 19:13:29.898646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mv5wh_51fa60a8-66ab-47f1-8b7e-ac798e054f4f/manager/0.log"