Jun 12 18:38:02.316577 ip-10-0-141-141 systemd[1]: Starting Kubernetes Kubelet... Jun 12 18:38:02.744484 ip-10-0-141-141 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 18:38:02.744484 ip-10-0-141-141 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 12 18:38:02.744484 ip-10-0-141-141 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 18:38:02.744484 ip-10-0-141-141 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 12 18:38:02.744484 ip-10-0-141-141 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 18:38:02.745915 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.745771 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 12 18:38:02.751001 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.750987 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.751001 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751001 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751005 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751009 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751013 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751015 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751018 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751021 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751025 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751029 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751044 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751048 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751051 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751054 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751057 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751060 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751063 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751066 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751068 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751071 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751074 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.751073 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751081 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751083 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751086 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751089 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751091 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751094 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751097 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751099 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751102 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751104 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751107 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751110 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751112 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751115 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751118 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751120 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751123 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751125 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751128 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751130 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.751601 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751133 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751135 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751138 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751140 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751143 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751146 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751150 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751153 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751156 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751158 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751162 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751164 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751167 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751170 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751174 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751177 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751181 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751183 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751186 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.752082 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751189 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751191 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751194 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751196 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751199 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751201 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751204 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751206 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751209 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751211 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751214 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751216 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751219 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751221 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751224 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751226 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751229 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751231 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751234 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751236 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.752555 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751239 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751241 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751244 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751246 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751249 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751252 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751254 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751259 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751261 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751264 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751266 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751269 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751272 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751275 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751278 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751280 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751283 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751285 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.753030 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.751289 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.754365 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754351 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754365 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754371 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754376 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754380 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754384 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754386 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754389 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754392 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754397 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754401 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754404 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754407 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.754403 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754410 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754412 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754415 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754418 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754421 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754423 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754426 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754428 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754432 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754435 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754437 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754440 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754443 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754447 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754451 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754468 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754471 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754474 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754477 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.754735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754479 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754482 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754484 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754487 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754490 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754493 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754496 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754499 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754501 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754504 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754506 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754509 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754511 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754514 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754516 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754524 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754527 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754532 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754537 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754541 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.755197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754546 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754550 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754554 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754557 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754560 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754563 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754566 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754568 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754571 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754573 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754576 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754578 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754582 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754584 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754587 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754590 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754592 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754595 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754597 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754600 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.755731 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754603 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754606 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754608 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754611 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754616 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754620 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754624 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754628 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754631 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754633 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754636 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754639 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754641 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754643 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754646 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754649 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754652 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754654 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754657 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754659 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.756240 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754663 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754666 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754668 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754671 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754673 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754676 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.754679 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755113 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755119 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755121 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755124 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755127 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755130 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755133 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755135 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755138 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755140 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755143 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755146 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755148 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.756736 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755151 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755153 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755156 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755158 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755161 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755163 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755166 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755169 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755172 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755174 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755178 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755182 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755186 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755190 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755194 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755198 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755201 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755205 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755209 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.757233 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755212 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755216 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755219 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755222 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755225 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755228 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755232 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755235 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755238 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755241 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755243 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755246 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755248 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755251 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755253 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755255 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755259 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755263 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755266 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.757735 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755272 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755277 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755281 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755283 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755286 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755288 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755291 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755293 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755295 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755298 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755300 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755303 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755305 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755320 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755323 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755325 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755328 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755331 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755333 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755336 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.758197 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755341 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755345 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755349 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755354 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755358 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755360 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755363 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755365 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755368 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755371 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755373 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755375 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755378 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755381 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755384 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755387 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755390 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755392 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755395 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755398 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.758690 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755400 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755403 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755405 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755407 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755410 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755413 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755416 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.755418 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755492 2577 flags.go:64] FLAG: --address="0.0.0.0" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755504 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755514 2577 flags.go:64] FLAG: --anonymous-auth="true" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755521 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755525 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755529 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755533 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755537 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755540 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755543 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755547 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755550 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755553 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755556 2577 flags.go:64] FLAG: --cgroup-root="" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755559 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 12 18:38:02.759211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755562 2577 flags.go:64] FLAG: --client-ca-file="" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755565 2577 flags.go:64] FLAG: --cloud-provider="external" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755568 2577 flags.go:64] FLAG: --cluster-dns="[]" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755573 2577 flags.go:64] FLAG: --cluster-domain="" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755576 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755580 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755583 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755589 2577 flags.go:64] FLAG: --container-log-max-files="5" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755594 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755599 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755604 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755608 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755611 2577 flags.go:64] FLAG: --contention-profiling="false" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755614 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755617 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755620 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755623 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755627 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755631 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755633 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755636 2577 flags.go:64] FLAG: --enable-load-reader="false" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755639 2577 flags.go:64] FLAG: --enable-server="true" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755642 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755647 2577 flags.go:64] FLAG: --event-burst="100" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755650 2577 flags.go:64] FLAG: --event-qps="50" Jun 12 18:38:02.759783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755652 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755655 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755658 2577 flags.go:64] FLAG: --eviction-hard="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755663 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755667 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755672 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755677 2577 flags.go:64] FLAG: --eviction-soft="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755681 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755685 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755689 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755692 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755695 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755698 2577 flags.go:64] FLAG: --fail-swap-on="true" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755701 2577 flags.go:64] FLAG: --feature-gates="" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755705 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755708 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755711 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755714 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755717 2577 flags.go:64] FLAG: --healthz-port="10248" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755720 2577 flags.go:64] FLAG: --help="false" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755723 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-141.ec2.internal" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755726 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755729 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jun 12 18:38:02.760399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755732 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755735 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755743 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755746 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755751 2577 flags.go:64] FLAG: --image-service-endpoint="" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755756 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755760 2577 flags.go:64] FLAG: --kube-api-burst="100" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755766 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755770 2577 flags.go:64] FLAG: --kube-api-qps="50" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755773 2577 flags.go:64] FLAG: --kube-reserved="" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755776 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755779 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755782 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755786 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755788 2577 flags.go:64] FLAG: --lock-file="" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755791 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755794 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755797 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755803 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755806 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755809 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755812 2577 flags.go:64] FLAG: --logging-format="text" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755815 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755819 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 12 18:38:02.760971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755822 2577 flags.go:64] FLAG: --manifest-url="" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755825 2577 flags.go:64] FLAG: --manifest-url-header="" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755831 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755836 2577 flags.go:64] FLAG: --max-open-files="1000000" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755842 2577 flags.go:64] FLAG: --max-pods="110" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755846 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755850 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755853 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755856 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755859 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755864 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755867 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755874 2577 flags.go:64] FLAG: --node-status-max-images="50" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755877 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755880 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755883 2577 flags.go:64] FLAG: --pod-cidr="" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755886 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755893 2577 flags.go:64] FLAG: --pod-manifest-path="" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755896 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755899 2577 flags.go:64] FLAG: --pods-per-core="0" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755902 2577 flags.go:64] FLAG: --port="10250" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755905 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755908 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0edf851498fcde5a6" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755913 2577 flags.go:64] FLAG: --qos-reserved="" Jun 12 18:38:02.761585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755918 2577 flags.go:64] FLAG: --read-only-port="10255" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755922 2577 flags.go:64] FLAG: --register-node="true" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755926 2577 flags.go:64] FLAG: --register-with-taints="" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755930 2577 flags.go:64] FLAG: --registry-burst="10" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755933 2577 flags.go:64] FLAG: --registry-qps="5" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755936 2577 flags.go:64] FLAG: --reserved-cpus="" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755938 2577 flags.go:64] FLAG: --reserved-memory="" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755942 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755946 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755949 2577 flags.go:64] FLAG: --rotate-certificates="false" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755952 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755954 2577 flags.go:64] FLAG: --runonce="false" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755957 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755960 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755963 2577 flags.go:64] FLAG: --seccomp-default="false" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755966 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755969 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755971 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755974 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755979 2577 flags.go:64] FLAG: --storage-driver-password="root" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755981 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755984 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755988 2577 flags.go:64] FLAG: --storage-driver-user="root" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755991 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.755996 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 12 18:38:02.762170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756002 2577 flags.go:64] FLAG: --system-cgroups="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756007 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756012 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756015 2577 flags.go:64] FLAG: --tls-cert-file="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756018 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756022 2577 flags.go:64] FLAG: --tls-min-version="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756025 2577 flags.go:64] FLAG: --tls-private-key-file="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756028 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756045 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756049 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756052 2577 flags.go:64] FLAG: --v="2" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756056 2577 flags.go:64] FLAG: --version="false" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756061 2577 flags.go:64] FLAG: --vmodule="" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756064 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.756068 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756190 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756195 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756198 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756201 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756204 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756207 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756210 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756212 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.762903 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756215 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756217 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756220 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756224 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756227 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756232 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756237 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756242 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756247 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756252 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756256 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756259 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756262 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756265 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756268 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756270 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756273 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756275 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756278 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756281 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.763630 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756284 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756286 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756289 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756291 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756294 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756297 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756300 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756303 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756305 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756321 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756326 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756330 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756334 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756336 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756339 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756343 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756347 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756350 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756353 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.764389 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756357 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756361 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756366 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756369 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756371 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756374 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756377 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756379 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756382 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756385 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756387 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756390 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756393 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756397 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756400 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756404 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756409 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756413 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756416 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756419 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.764866 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756422 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756425 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756427 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756430 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756433 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756435 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756438 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756440 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756444 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756447 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756449 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756452 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756454 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756457 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756461 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756463 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756466 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756469 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756471 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.765364 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756474 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756478 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756482 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756487 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756491 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756495 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756498 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756500 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756503 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756506 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756508 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756510 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.756513 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.757069 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.765196 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 12 18:38:02.765833 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.765211 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.765959 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.765972 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766029 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766035 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766038 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766042 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766045 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766048 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766051 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766054 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766056 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766059 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766061 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766064 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766067 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766069 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766072 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766075 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.766216 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766079 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766082 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766086 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766090 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766093 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766096 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766099 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766102 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766105 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766107 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766110 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766112 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766115 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766117 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766120 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766123 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766126 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766129 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766132 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.766854 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766134 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766137 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766140 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766143 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766145 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766148 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766150 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766153 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766156 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766158 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766161 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766164 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766167 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766169 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766172 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766174 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766177 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766180 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766182 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766185 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.767357 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766188 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766191 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766193 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766196 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766198 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766201 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766203 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766206 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766209 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766212 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766215 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766218 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766221 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766224 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766226 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766229 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766233 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766236 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766239 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766243 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.767837 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766245 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766248 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766251 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766254 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766257 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766260 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766263 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766265 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766268 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766271 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766274 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766276 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766279 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766282 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766284 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766287 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766290 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766292 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766295 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.768327 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766298 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766300 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766303 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766305 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766321 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.766327 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766444 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766450 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766453 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766456 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766460 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766464 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766467 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766470 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 18:38:02.768830 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766474 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766477 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766481 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766484 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766488 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766491 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766494 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766497 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766500 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766502 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766506 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766508 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766511 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766514 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766517 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766520 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766523 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766525 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766528 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766530 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 18:38:02.769188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766533 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766535 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766538 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766540 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766543 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766545 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766548 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766550 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766553 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766555 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766558 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766560 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766563 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766565 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766568 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766570 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766573 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766576 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766579 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766582 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 18:38:02.769688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766584 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766587 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766589 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766592 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766594 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766597 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766599 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766602 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766605 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766607 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766609 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766612 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766615 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766617 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766620 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766622 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766624 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766627 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766629 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 18:38:02.770153 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766632 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766634 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766637 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766639 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766642 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766644 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766647 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766649 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766652 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766655 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766658 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766661 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766664 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766666 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766669 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766671 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766674 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766676 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766679 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766681 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 18:38:02.770636 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766684 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766686 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766689 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766691 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766693 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766696 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766698 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766701 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766703 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766706 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766709 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:02.766711 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.766716 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.766829 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.768960 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 12 18:38:02.771124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.770397 2577 server.go:1019] "Starting client certificate rotation" Jun 12 18:38:02.771515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.770493 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 12 18:38:02.771515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.770529 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 12 18:38:02.772905 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.772892 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 12 18:38:02.775852 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.775837 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 12 18:38:02.783683 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.783666 2577 log.go:25] "Validated CRI v1 runtime API" Jun 12 18:38:02.788334 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.788305 2577 log.go:25] "Validated CRI v1 image API" Jun 12 18:38:02.790258 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.790238 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 12 18:38:02.793756 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.793735 2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ab42462b-fcfe-4ad1-b566-f6e8662bc061:/dev/nvme0n1p4 f20df309-72e9-450d-8596-1ee5d81c6ce3:/dev/nvme0n1p3] Jun 12 18:38:02.793820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.793754 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 12 18:38:02.799054 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.799026 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 12 18:38:02.799356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.799214 2577 manager.go:216] Machine: {Timestamp:2026-06-12 18:38:02.797829972 +0000 UTC m=+0.364107197 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099677 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d61cc71be8ad20055036fb0127540 SystemUUID:ec2d61cc-71be-8ad2-0055-036fb0127540 BootID:0d85193e-be07-460f-ad3a-d81a92829ed6 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b5:dc:55:2e:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b5:dc:55:2e:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:a3:d3:17:47:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 12 18:38:02.799356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.799343 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 12 18:38:02.799480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.799423 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 12 18:38:02.802287 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.802259 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 12 18:38:02.802450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.802297 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 12 18:38:02.803321 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.803301 2577 topology_manager.go:138] "Creating topology manager with none policy" Jun 12 18:38:02.803357 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.803323 2577 container_manager_linux.go:309] "Creating device plugin manager" Jun 12 18:38:02.803357 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.803331 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 12 18:38:02.803998 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.803987 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 12 18:38:02.805259 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.805250 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 12 18:38:02.808079 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808067 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 12 18:38:02.808196 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808187 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 12 18:38:02.808255 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808246 2577 kubelet.go:486] "Attempting to sync node with API server" Jun 12 18:38:02.808301 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808288 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 12 18:38:02.808392 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808354 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 12 18:38:02.808392 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808367 2577 kubelet.go:398] "Adding apiserver pod source" Jun 12 18:38:02.808392 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808379 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 12 18:38:02.808964 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808940 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 12 18:38:02.809042 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.808972 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 12 18:38:02.812808 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.812789 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 12 18:38:02.814185 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814171 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 12 18:38:02.814240 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814200 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 12 18:38:02.814375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814366 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 12 18:38:02.814416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814380 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 12 18:38:02.814416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814389 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 12 18:38:02.814416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814397 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 12 18:38:02.814416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814405 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 12 18:38:02.814416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814411 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 12 18:38:02.814906 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814896 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 12 18:38:02.814940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814907 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 12 18:38:02.814940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814915 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 12 18:38:02.814940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814920 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 12 18:38:02.814940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814929 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 12 18:38:02.814940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.814937 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 12 18:38:02.815718 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.815707 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 12 18:38:02.815751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.815720 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 12 18:38:02.818632 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.818611 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 12 18:38:02.818632 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.818617 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 12 18:38:02.818743 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.818657 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 12 18:38:02.819005 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.818992 2577 server.go:1268] "Started kubelet" Jun 12 18:38:02.819107 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.819080 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 12 18:38:02.819156 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.819078 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 12 18:38:02.819156 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.819135 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 12 18:38:02.819808 ip-10-0-141-141 systemd[1]: Started Kubernetes Kubelet. Jun 12 18:38:02.820727 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.820646 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 12 18:38:02.821246 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.821234 2577 server.go:310] "Adding debug handlers to kubelet server" Jun 12 18:38:02.821936 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.821913 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9wcs" Jun 12 18:38:02.825177 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.824013 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-141.ec2.internal.18b86958d1902a3a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-141.ec2.internal,UID:ip-10-0-141-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-141.ec2.internal,},FirstTimestamp:2026-06-12 18:38:02.818955834 +0000 UTC m=+0.385233065,LastTimestamp:2026-06-12 18:38:02.818955834 +0000 UTC m=+0.385233065,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-141.ec2.internal,}" Jun 12 18:38:02.826629 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.826613 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 12 18:38:02.826730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.826636 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 12 18:38:02.826860 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.826839 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:02.827011 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.826991 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 12 18:38:02.827391 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.827373 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 12 18:38:02.827391 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.827394 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 12 18:38:02.827538 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.827378 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 12 18:38:02.827538 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.827501 2577 reconstruct.go:97] "Volume reconstruction finished" Jun 12 18:38:02.827538 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.827509 2577 reconciler.go:29] "Reconciler: start to sync state" Jun 12 18:38:02.829587 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.829569 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9wcs" Jun 12 18:38:02.830069 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830048 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 12 18:38:02.830069 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830065 2577 factory.go:55] Registering systemd factory Jun 12 18:38:02.830169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830074 2577 factory.go:223] Registration of the systemd container factory successfully Jun 12 18:38:02.830295 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830267 2577 factory.go:153] Registering CRI-O factory Jun 12 18:38:02.830295 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830281 2577 factory.go:223] Registration of the crio container factory successfully Jun 12 18:38:02.830396 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830299 2577 factory.go:103] Registering Raw factory Jun 12 18:38:02.830396 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.830324 2577 manager.go:1196] Started watching for new ooms in manager Jun 12 18:38:02.831292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.831004 2577 manager.go:349] Starting recovery of all containers Jun 12 18:38:02.838858 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.838830 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 12 18:38:02.839216 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.839167 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 12 18:38:02.841746 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.841589 2577 manager.go:354] Recovery completed Jun 12 18:38:02.846161 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.846148 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.848011 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.847994 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.848081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848025 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.848081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848036 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.848560 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848545 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 12 18:38:02.848560 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848559 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 12 18:38:02.848648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848579 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 12 18:38:02.848648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.848560 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.849324 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849300 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.849377 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849337 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.849377 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849348 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.849810 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849799 2577 policy_none.go:49] "None policy: Start" Jun 12 18:38:02.849843 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849813 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 12 18:38:02.849843 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.849823 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 12 18:38:02.851422 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.851407 2577 policy_none.go:47] "Start" Jun 12 18:38:02.890902 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.890886 2577 manager.go:337] "Starting Device Plugin manager" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.890916 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.890926 2577 server.go:87] "Starting device plugin registration server" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891168 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891179 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891287 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891400 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891410 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.891719 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.891981 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.892032 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.893167 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.893195 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.893216 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:02.893249 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 12 18:38:02.902595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.895373 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 18:38:02.991954 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.991915 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.992969 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.992951 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.993043 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.992983 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.993043 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.992997 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.993120 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.993046 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-141.ec2.internal" Jun 12 18:38:02.993328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.993294 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal"] Jun 12 18:38:02.993377 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.993368 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.994076 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.994060 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.994158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.994088 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.994158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.994100 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.995370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.995329 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.995487 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.995428 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:02.995487 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.995453 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.995992 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.995978 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.996064 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.996012 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.996064 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.996027 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.996064 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.995980 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.996210 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.996075 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.996210 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.996085 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:02.997458 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.997443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:02.997533 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.997465 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 18:38:02.998084 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.998069 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientMemory" Jun 12 18:38:02.998158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.998092 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 18:38:02.998158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:02.998101 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeHasSufficientPID" Jun 12 18:38:03.001827 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.001809 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.001914 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.001835 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-141.ec2.internal\": node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.030085 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.030062 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.036081 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.036059 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-141.ec2.internal\" not found" node="ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.040530 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.040511 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-141.ec2.internal\" not found" node="ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.129398 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.129366 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ff5b698212972d9b24580e1a2f13644-config\") pod \"kube-apiserver-proxy-ip-10-0-141-141.ec2.internal\" (UID: \"4ff5b698212972d9b24580e1a2f13644\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.129398 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.129399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.129599 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.129422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.130381 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.130362 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.229902 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.229868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.229902 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.229902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ff5b698212972d9b24580e1a2f13644-config\") pod \"kube-apiserver-proxy-ip-10-0-141-141.ec2.internal\" (UID: \"4ff5b698212972d9b24580e1a2f13644\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.230097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.229920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.230097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.229966 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.230097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.230046 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ff5b698212972d9b24580e1a2f13644-config\") pod \"kube-apiserver-proxy-ip-10-0-141-141.ec2.internal\" (UID: \"4ff5b698212972d9b24580e1a2f13644\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.230097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.230051 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41fc3776e4bca81320bee88c10e1cb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal\" (UID: \"41fc3776e4bca81320bee88c10e1cb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.230918 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.230900 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.331714 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.331647 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.338814 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.338795 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.343994 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.343977 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.432642 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.432603 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.533035 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.533009 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.633491 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.633472 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.734040 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.734013 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.770197 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.770172 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 12 18:38:03.770749 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.770337 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 12 18:38:03.798171 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.798146 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 18:38:03.834919 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:03.834894 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-141.ec2.internal\" not found" Jun 12 18:38:03.840049 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.840028 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-11 18:33:02 +0000 UTC" deadline="2028-02-17 19:14:07.669605933 +0000 UTC" Jun 12 18:38:03.840101 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.840053 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14760h36m3.829558343s" Jun 12 18:38:03.864293 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:03.864255 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41fc3776e4bca81320bee88c10e1cb3f.slice/crio-d40ff7c7d7dea63039980022945bc540c2ce3921320d657f4479c25b28219544 WatchSource:0}: Error finding container d40ff7c7d7dea63039980022945bc540c2ce3921320d657f4479c25b28219544: Status 404 returned error can't find the container with id d40ff7c7d7dea63039980022945bc540c2ce3921320d657f4479c25b28219544 Jun 12 18:38:03.864565 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:03.864529 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff5b698212972d9b24580e1a2f13644.slice/crio-78d51adc360cf6ab6ef39932db5dda743787faa760a0d3b1587f8fc5db9b7a1c WatchSource:0}: Error finding container 78d51adc360cf6ab6ef39932db5dda743787faa760a0d3b1587f8fc5db9b7a1c: Status 404 returned error can't find the container with id 78d51adc360cf6ab6ef39932db5dda743787faa760a0d3b1587f8fc5db9b7a1c Jun 12 18:38:03.872154 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.872139 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 18:38:03.897094 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.897028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" event={"ID":"4ff5b698212972d9b24580e1a2f13644","Type":"ContainerStarted","Data":"78d51adc360cf6ab6ef39932db5dda743787faa760a0d3b1587f8fc5db9b7a1c"} Jun 12 18:38:03.897955 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.897937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" event={"ID":"41fc3776e4bca81320bee88c10e1cb3f","Type":"ContainerStarted","Data":"d40ff7c7d7dea63039980022945bc540c2ce3921320d657f4479c25b28219544"} Jun 12 18:38:03.907965 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.907946 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 18:38:03.927460 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.927445 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.938463 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.938445 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 12 18:38:03.940404 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.940393 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" Jun 12 18:38:03.951549 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:03.951533 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 12 18:38:04.351593 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.351520 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 18:38:04.810475 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.810442 2577 apiserver.go:52] "Watching apiserver" Jun 12 18:38:04.818897 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.818872 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 12 18:38:04.819269 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.819246 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-g7xz5","kube-system/global-pull-secret-syncer-dkcrw","kube-system/konnectivity-agent-8rl5k","openshift-dns/node-resolver-hc62t","openshift-multus/multus-additional-cni-plugins-vdh88","openshift-network-operator/iptables-alerter-6ljbd","openshift-ovn-kubernetes/ovnkube-node-kdvng","kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw","openshift-cluster-node-tuning-operator/tuned-p4q5c","openshift-image-registry/node-ca-z67bx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal","openshift-multus/multus-s9hn6","openshift-multus/network-metrics-daemon-8k97v"] Jun 12 18:38:04.820831 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.820812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:04.820913 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.820884 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:04.821967 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.821936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.822042 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.822003 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:04.823084 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.823063 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.824416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.824392 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.825648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.825628 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 12 18:38:04.825818 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.825802 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 12 18:38:04.825908 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.825892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zhqd4\"" Jun 12 18:38:04.826099 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.826076 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.826987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.826835 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.826987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.826863 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.826987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.826866 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9s6dh\"" Jun 12 18:38:04.827192 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.827133 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.827716 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.827698 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 12 18:38:04.828387 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.828304 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 12 18:38:04.828518 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.828500 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.828757 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.828612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.829072 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.829038 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.829274 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.829256 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 12 18:38:04.829511 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.829496 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 12 18:38:04.829967 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.829824 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 12 18:38:04.829967 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.829856 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.830258 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.830233 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.830865 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.830630 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.830865 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.830682 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d8767\"" Jun 12 18:38:04.830865 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.830709 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zcm6k\"" Jun 12 18:38:04.832152 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.831778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.832152 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.831953 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.832152 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832007 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.832355 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832164 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 12 18:38:04.832704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832685 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8cbdr\"" Jun 12 18:38:04.832785 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832742 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 12 18:38:04.832785 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832763 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 12 18:38:04.832886 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.832838 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 12 18:38:04.833028 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.833000 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.833115 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.833047 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4wbmx\"" Jun 12 18:38:04.833115 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.833102 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 12 18:38:04.833245 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.833231 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.833384 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.833366 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:04.834749 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.834724 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.836472 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.835733 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.836472 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.835785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nm9qw\"" Jun 12 18:38:04.836472 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.835856 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 12 18:38:04.837879 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837744 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qxrsv\"" Jun 12 18:38:04.837879 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837813 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 12 18:38:04.837879 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837826 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837929 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlfzm\" (UniqueName: \"kubernetes.io/projected/280a69e5-2a94-475f-9444-3b61d6fa80a9-kube-api-access-tlfzm\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.837998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838024 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-etc-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-bin\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98db3c71-979c-4f56-ab78-577193af446f-host-slash\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-system-cni-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cnibin\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4zgw\" (UniqueName: \"kubernetes.io/projected/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-kube-api-access-z4zgw\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-netns\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-log-socket\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6661090-b2a0-4017-8029-bf18a56e6039-konnectivity-ca\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/280a69e5-2a94-475f-9444-3b61d6fa80a9-hosts-file\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-var-lib-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838473 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-netd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.838509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lrkw\" (UniqueName: \"kubernetes.io/projected/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-kube-api-access-5lrkw\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98db3c71-979c-4f56-ab78-577193af446f-iptables-alerter-script\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-os-release\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-kubelet\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-slash\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-config\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6661090-b2a0-4017-8029-bf18a56e6039-agent-certs\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838773 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-systemd-units\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s28wt\" (UniqueName: \"kubernetes.io/projected/98db3c71-979c-4f56-ab78-577193af446f-kube-api-access-s28wt\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovn-node-metrics-cert\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-dbus\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.838982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-ovn\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280a69e5-2a94-475f-9444-3b61d6fa80a9-tmp-dir\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.839176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839047 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839063 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-systemd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-node-log\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.839116 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-env-overrides\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-script-lib\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.839898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.839238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-kubelet-config\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.840327 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.840290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 12 18:38:04.840506 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.840297 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6hk5r\"" Jun 12 18:38:04.841068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.841048 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 12 18:38:04.859252 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.859230 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bd6db" Jun 12 18:38:04.867154 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.867138 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bd6db" Jun 12 18:38:04.929054 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.929032 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 12 18:38:04.940319 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-systemd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-node-log\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-env-overrides\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940379 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-systemd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-script-lib\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-tmp\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxg4l\" (UniqueName: \"kubernetes.io/projected/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-kube-api-access-qxg4l\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-node-log\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-cnibin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-kubelet-config\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlfzm\" (UniqueName: \"kubernetes.io/projected/280a69e5-2a94-475f-9444-3b61d6fa80a9-kube-api-access-tlfzm\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysconfig\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.940690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-multus-certs\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqr2\" (UniqueName: \"kubernetes.io/projected/92c90488-bdc9-427f-b43c-e3dec7f332e2-kube-api-access-lrqr2\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940724 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-netd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-conf\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cnibin\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-netns\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940812 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-netd\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkg52\" (UniqueName: \"kubernetes.io/projected/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-kube-api-access-fkg52\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940864 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cnibin\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-socket-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940883 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-netns\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/280a69e5-2a94-475f-9444-3b61d6fa80a9-hosts-file\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940907 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-env-overrides\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-serviceca\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940944 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/280a69e5-2a94-475f-9444-3b61d6fa80a9-hosts-file\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.941111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940910 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98db3c71-979c-4f56-ab78-577193af446f-iptables-alerter-script\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.940987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-slash\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941011 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-hostroot\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941034 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-script-lib\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-tuned\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941094 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-slash\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941135 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-kubelet-config\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941263 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98db3c71-979c-4f56-ab78-577193af446f-iptables-alerter-script\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.941721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941623 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-multus\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-kubelet\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-sys-fs\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s28wt\" (UniqueName: \"kubernetes.io/projected/98db3c71-979c-4f56-ab78-577193af446f-kube-api-access-s28wt\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.942149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.941905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovn-node-metrics-cert\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942405 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-host\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.942405 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942355 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.942523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-cni-binary-copy\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-dbus\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.942523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-host\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:04.942523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-bin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280a69e5-2a94-475f-9444-3b61d6fa80a9-tmp-dir\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942522 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e14573a-c737-43c5-bd00-07679062a688-dbus\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942631 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-etc-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942628 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-bin\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942680 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-os-release\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-etc-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lrkw\" (UniqueName: \"kubernetes.io/projected/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-kube-api-access-5lrkw\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942716 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-cni-bin\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98db3c71-979c-4f56-ab78-577193af446f-host-slash\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-system-cni-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.942764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98db3c71-979c-4f56-ab78-577193af446f-host-slash\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942796 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-system-cni-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4zgw\" (UniqueName: \"kubernetes.io/projected/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-kube-api-access-z4zgw\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942844 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280a69e5-2a94-475f-9444-3b61d6fa80a9-tmp-dir\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-log-socket\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942880 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-log-socket\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-conf-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-daemon-config\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.942991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6661090-b2a0-4017-8029-bf18a56e6039-konnectivity-ca\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-var-lib-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-lib-modules\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943115 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-var-lib-openvswitch\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.943385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943123 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-os-release\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-kubelet\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-config\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943617 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-systemd\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-sys\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-var-lib-kubelet\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-device-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6661090-b2a0-4017-8029-bf18a56e6039-agent-certs\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-systemd-units\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-netns\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-registration-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.943877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8mlz\" (UniqueName: \"kubernetes.io/projected/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kube-api-access-n8mlz\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.944118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6661090-b2a0-4017-8029-bf18a56e6039-konnectivity-ca\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.944143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.944142 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-os-release\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.944766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.944277 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-modprobe-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.944766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.944364 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-host-kubelet\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.944862 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.944831 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovnkube-config\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.944949 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.944932 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:04.945041 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.945030 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:05.444990571 +0000 UTC m=+3.011267800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:04.945766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-systemd-units\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.945766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-kubernetes\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.945920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.945920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-system-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.945920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-etc-kubernetes\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.945920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnnl\" (UniqueName: \"kubernetes.io/projected/dd0084c5-7e69-4534-8301-20b771c71427-kube-api-access-9qnnl\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-ovn\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.945974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-run\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946002 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-socket-dir-parent\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-k8s-cni-cncf-io\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946120 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-run-ovn\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.946149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946131 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-ovn-node-metrics-cert\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:04.946657 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.946634 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.948440 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.948414 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6661090-b2a0-4017-8029-bf18a56e6039-agent-certs\") pod \"konnectivity-agent-8rl5k\" (UID: \"b6661090-b2a0-4017-8029-bf18a56e6039\") " pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:04.949426 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.949376 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlfzm\" (UniqueName: \"kubernetes.io/projected/280a69e5-2a94-475f-9444-3b61d6fa80a9-kube-api-access-tlfzm\") pod \"node-resolver-hc62t\" (UID: \"280a69e5-2a94-475f-9444-3b61d6fa80a9\") " pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:04.954236 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.954214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s28wt\" (UniqueName: \"kubernetes.io/projected/98db3c71-979c-4f56-ab78-577193af446f-kube-api-access-s28wt\") pod \"iptables-alerter-6ljbd\" (UID: \"98db3c71-979c-4f56-ab78-577193af446f\") " pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:04.955432 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.955254 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:04.955432 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.955278 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:04.955432 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.955290 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:04.955432 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:04.955349 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:05.455335812 +0000 UTC m=+3.021613039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:04.957567 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.957546 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4zgw\" (UniqueName: \"kubernetes.io/projected/96fd2d13-f5b8-4cc1-9192-a90e13ba3df1-kube-api-access-z4zgw\") pod \"multus-additional-cni-plugins-vdh88\" (UID: \"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1\") " pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:04.957723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:04.957697 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lrkw\" (UniqueName: \"kubernetes.io/projected/b5f5bbfa-6d5d-4005-85f0-89303c4cc27e-kube-api-access-5lrkw\") pod \"ovnkube-node-kdvng\" (UID: \"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:05.046965 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.046932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-registration-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.046986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8mlz\" (UniqueName: \"kubernetes.io/projected/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kube-api-access-n8mlz\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-modprobe-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-kubernetes\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-system-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047094 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-registration-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-etc-kubernetes\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047124 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-kubernetes\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnnl\" (UniqueName: \"kubernetes.io/projected/dd0084c5-7e69-4534-8301-20b771c71427-kube-api-access-9qnnl\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-run\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047165 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-modprobe-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047185 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-system-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-socket-dir-parent\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047201 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-d\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-k8s-cni-cncf-io\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-socket-dir-parent\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047241 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-etc-kubernetes\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-tmp\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxg4l\" (UniqueName: \"kubernetes.io/projected/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-kube-api-access-qxg4l\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047374 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-k8s-cni-cncf-io\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047419 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-run\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047483 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-cnibin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysconfig\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-multus-certs\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqr2\" (UniqueName: \"kubernetes.io/projected/92c90488-bdc9-427f-b43c-e3dec7f332e2-kube-api-access-lrqr2\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047580 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysconfig\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-conf\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047568 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-cnibin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.047673 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047595 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-multus-certs\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047690 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-sysctl-conf\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkg52\" (UniqueName: \"kubernetes.io/projected/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-kube-api-access-fkg52\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047724 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-socket-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-serviceca\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-hostroot\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-tuned\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-multus\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-kubelet\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-hostroot\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-socket-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047893 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-multus\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-sys-fs\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-host\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.047936 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047929 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-kubelet\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-cni-binary-copy\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.048516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-host\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.047997 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-sys-fs\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.048018 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:05.547983591 +0000 UTC m=+3.114260824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048021 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-host\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048044 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-host\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-bin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-os-release\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-conf-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048204 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-cni-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-daemon-config\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048198 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-var-lib-cni-bin\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048228 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048254 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-serviceca\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048264 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-os-release\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048254 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-conf-dir\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.049335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-lib-modules\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-systemd\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-sys\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-var-lib-kubelet\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048417 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-lib-modules\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-device-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-systemd\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-device-dir\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-netns\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048531 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-sys\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048549 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c90488-bdc9-427f-b43c-e3dec7f332e2-host-run-netns\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048579 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-var-lib-kubelet\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048634 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-cni-binary-copy\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.048669 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c90488-bdc9-427f-b43c-e3dec7f332e2-multus-daemon-config\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.050085 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.049830 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-tmp\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.050760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.050341 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-etc-tuned\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.059058 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.059027 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxg4l\" (UniqueName: \"kubernetes.io/projected/9a57803a-5b1f-4507-ba89-7059b1a7ce7e-kube-api-access-qxg4l\") pod \"tuned-p4q5c\" (UID: \"9a57803a-5b1f-4507-ba89-7059b1a7ce7e\") " pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.060155 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.060133 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkg52\" (UniqueName: \"kubernetes.io/projected/9e5ce852-849e-4d39-bcbf-d6d44a3bdc36-kube-api-access-fkg52\") pod \"node-ca-z67bx\" (UID: \"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36\") " pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.060468 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.060446 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnnl\" (UniqueName: \"kubernetes.io/projected/dd0084c5-7e69-4534-8301-20b771c71427-kube-api-access-9qnnl\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:05.060584 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.060530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqr2\" (UniqueName: \"kubernetes.io/projected/92c90488-bdc9-427f-b43c-e3dec7f332e2-kube-api-access-lrqr2\") pod \"multus-s9hn6\" (UID: \"92c90488-bdc9-427f-b43c-e3dec7f332e2\") " pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.060760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.060733 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8mlz\" (UniqueName: \"kubernetes.io/projected/e888a720-68b1-47e8-b4e5-0bfb1e1454a6-kube-api-access-n8mlz\") pod \"aws-ebs-csi-driver-node-p8pxw\" (UID: \"e888a720-68b1-47e8-b4e5-0bfb1e1454a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.102175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.102148 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 18:38:05.136686 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.136659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:05.143764 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.143740 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hc62t" Jun 12 18:38:05.149859 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.149842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vdh88" Jun 12 18:38:05.157751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.157732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6ljbd" Jun 12 18:38:05.163377 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.163354 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:05.169936 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.169920 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" Jun 12 18:38:05.177439 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.177422 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" Jun 12 18:38:05.185074 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.185050 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z67bx" Jun 12 18:38:05.190603 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.190581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9hn6" Jun 12 18:38:05.450492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.450462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:05.450669 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.450598 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:05.450669 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.450667 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:06.45064843 +0000 UTC m=+4.016925658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:05.551808 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.551770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:05.551996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.551868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:05.551996 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.551950 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:05.551996 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.551984 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:05.551996 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.551990 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:05.552201 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.552057 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:06.552037157 +0000 UTC m=+4.118314375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:05.552201 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.551994 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:05.552201 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:05.552129 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:06.552112231 +0000 UTC m=+4.118389446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:05.773828 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.773184 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5ce852_849e_4d39_bcbf_d6d44a3bdc36.slice/crio-f39a5940efc37048c01d9b6c5d983f733ea3722b5d188ad076655b7174ef82ea WatchSource:0}: Error finding container f39a5940efc37048c01d9b6c5d983f733ea3722b5d188ad076655b7174ef82ea: Status 404 returned error can't find the container with id f39a5940efc37048c01d9b6c5d983f733ea3722b5d188ad076655b7174ef82ea Jun 12 18:38:05.775075 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.775007 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode888a720_68b1_47e8_b4e5_0bfb1e1454a6.slice/crio-4d59cf51fc919ad18082649ca3d02fd3e483feb9008ec48e37a76592734754b2 WatchSource:0}: Error finding container 4d59cf51fc919ad18082649ca3d02fd3e483feb9008ec48e37a76592734754b2: Status 404 returned error can't find the container with id 4d59cf51fc919ad18082649ca3d02fd3e483feb9008ec48e37a76592734754b2 Jun 12 18:38:05.775930 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.775903 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c90488_bdc9_427f_b43c_e3dec7f332e2.slice/crio-daf1a1b3cce62d46fad77b48abb316d0c452b5a45588683b65e577ec1a3cef7a WatchSource:0}: Error finding container daf1a1b3cce62d46fad77b48abb316d0c452b5a45588683b65e577ec1a3cef7a: Status 404 returned error can't find the container with id daf1a1b3cce62d46fad77b48abb316d0c452b5a45588683b65e577ec1a3cef7a Jun 12 18:38:05.780523 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.780466 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fd2d13_f5b8_4cc1_9192_a90e13ba3df1.slice/crio-6f38d053a3cf339bc1d562dccf375eeff0820bd19fe032354a28aa2b1a43cde5 WatchSource:0}: Error finding container 6f38d053a3cf339bc1d562dccf375eeff0820bd19fe032354a28aa2b1a43cde5: Status 404 returned error can't find the container with id 6f38d053a3cf339bc1d562dccf375eeff0820bd19fe032354a28aa2b1a43cde5 Jun 12 18:38:05.781757 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.781736 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a57803a_5b1f_4507_ba89_7059b1a7ce7e.slice/crio-6ff3b97d8d9ecd533ea4e9882dbb212056367c7507cc34363b5cb3e080ecbc83 WatchSource:0}: Error finding container 6ff3b97d8d9ecd533ea4e9882dbb212056367c7507cc34363b5cb3e080ecbc83: Status 404 returned error can't find the container with id 6ff3b97d8d9ecd533ea4e9882dbb212056367c7507cc34363b5cb3e080ecbc83 Jun 12 18:38:05.783002 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.782979 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6661090_b2a0_4017_8029_bf18a56e6039.slice/crio-808e7f46ecc0446bd6c0d3c04988a78a14760b0e8af009340cffe57a0220466f WatchSource:0}: Error finding container 808e7f46ecc0446bd6c0d3c04988a78a14760b0e8af009340cffe57a0220466f: Status 404 returned error can't find the container with id 808e7f46ecc0446bd6c0d3c04988a78a14760b0e8af009340cffe57a0220466f Jun 12 18:38:05.784047 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.783911 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f5bbfa_6d5d_4005_85f0_89303c4cc27e.slice/crio-e382806dadbd62701cd80da545d0510e3a3d953efe646f0ec7dde0564cdd01f8 WatchSource:0}: Error finding container e382806dadbd62701cd80da545d0510e3a3d953efe646f0ec7dde0564cdd01f8: Status 404 returned error can't find the container with id e382806dadbd62701cd80da545d0510e3a3d953efe646f0ec7dde0564cdd01f8 Jun 12 18:38:05.784562 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.784517 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280a69e5_2a94_475f_9444_3b61d6fa80a9.slice/crio-f05a94d775aa38e107e6fab177aa292a6923e812b9db69f5b1b6490a095796dd WatchSource:0}: Error finding container f05a94d775aa38e107e6fab177aa292a6923e812b9db69f5b1b6490a095796dd: Status 404 returned error can't find the container with id f05a94d775aa38e107e6fab177aa292a6923e812b9db69f5b1b6490a095796dd Jun 12 18:38:05.785566 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:05.785547 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98db3c71_979c_4f56_ab78_577193af446f.slice/crio-ee1057a3f35b8dea8f0aa32ae4356e58eb1bdfdf57906f6ada2f3a8865f0581a WatchSource:0}: Error finding container ee1057a3f35b8dea8f0aa32ae4356e58eb1bdfdf57906f6ada2f3a8865f0581a: Status 404 returned error can't find the container with id ee1057a3f35b8dea8f0aa32ae4356e58eb1bdfdf57906f6ada2f3a8865f0581a Jun 12 18:38:05.869612 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.869555 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-11 18:33:04 +0000 UTC" deadline="2028-02-13 22:13:44.69985499 +0000 UTC" Jun 12 18:38:05.869612 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.869606 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14667h35m38.830253368s" Jun 12 18:38:05.901117 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.901086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6ljbd" event={"ID":"98db3c71-979c-4f56-ab78-577193af446f","Type":"ContainerStarted","Data":"ee1057a3f35b8dea8f0aa32ae4356e58eb1bdfdf57906f6ada2f3a8865f0581a"} Jun 12 18:38:05.902183 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.902155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hc62t" event={"ID":"280a69e5-2a94-475f-9444-3b61d6fa80a9","Type":"ContainerStarted","Data":"f05a94d775aa38e107e6fab177aa292a6923e812b9db69f5b1b6490a095796dd"} Jun 12 18:38:05.903281 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.903258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9hn6" event={"ID":"92c90488-bdc9-427f-b43c-e3dec7f332e2","Type":"ContainerStarted","Data":"daf1a1b3cce62d46fad77b48abb316d0c452b5a45588683b65e577ec1a3cef7a"} Jun 12 18:38:05.904723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.904702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" event={"ID":"e888a720-68b1-47e8-b4e5-0bfb1e1454a6","Type":"ContainerStarted","Data":"4d59cf51fc919ad18082649ca3d02fd3e483feb9008ec48e37a76592734754b2"} Jun 12 18:38:05.905841 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.905816 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"e382806dadbd62701cd80da545d0510e3a3d953efe646f0ec7dde0564cdd01f8"} Jun 12 18:38:05.906930 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.906899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rl5k" event={"ID":"b6661090-b2a0-4017-8029-bf18a56e6039","Type":"ContainerStarted","Data":"808e7f46ecc0446bd6c0d3c04988a78a14760b0e8af009340cffe57a0220466f"} Jun 12 18:38:05.907940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.907920 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" event={"ID":"9a57803a-5b1f-4507-ba89-7059b1a7ce7e","Type":"ContainerStarted","Data":"6ff3b97d8d9ecd533ea4e9882dbb212056367c7507cc34363b5cb3e080ecbc83"} Jun 12 18:38:05.908974 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.908951 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerStarted","Data":"6f38d053a3cf339bc1d562dccf375eeff0820bd19fe032354a28aa2b1a43cde5"} Jun 12 18:38:05.910170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.910130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z67bx" event={"ID":"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36","Type":"ContainerStarted","Data":"f39a5940efc37048c01d9b6c5d983f733ea3722b5d188ad076655b7174ef82ea"} Jun 12 18:38:05.912217 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.912188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" event={"ID":"4ff5b698212972d9b24580e1a2f13644","Type":"ContainerStarted","Data":"adbd217cbe8f978f70c30e9385aa574dff0cdc9332909b9c531deebffe45668c"} Jun 12 18:38:05.936686 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:05.936644 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-141.ec2.internal" podStartSLOduration=2.936628727 podStartE2EDuration="2.936628727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:05.936270955 +0000 UTC m=+3.502548191" watchObservedRunningTime="2026-06-12 18:38:05.936628727 +0000 UTC m=+3.502905976" Jun 12 18:38:06.460697 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.460660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:06.460895 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.460806 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:06.460895 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.460868 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:08.460851642 +0000 UTC m=+6.027128869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:06.561725 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.561687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:06.561888 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.561785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:06.561958 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.561920 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:06.562010 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.561979 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:08.561961326 +0000 UTC m=+6.128238561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:06.562431 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.562411 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:06.562526 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.562436 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:06.562526 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.562451 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:06.562526 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.562495 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:08.562480459 +0000 UTC m=+6.128757676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:06.870731 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.870644 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-11 18:33:04 +0000 UTC" deadline="2028-03-05 05:04:08.648354362 +0000 UTC" Jun 12 18:38:06.870731 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.870683 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15154h26m1.777675465s" Jun 12 18:38:06.896354 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.896306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:06.896516 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.896472 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:06.897221 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.896992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:06.897221 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.897091 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:06.897221 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.896992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:06.897221 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:06.897184 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:06.928045 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.927420 2577 generic.go:356] "Generic (PLEG): container finished" podID="41fc3776e4bca81320bee88c10e1cb3f" containerID="2e74c1f5b3ff1bac3cc26a34ef546907b68a64c3b9ca828bd37747c941505108" exitCode=0 Jun 12 18:38:06.928045 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:06.927499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" event={"ID":"41fc3776e4bca81320bee88c10e1cb3f","Type":"ContainerDied","Data":"2e74c1f5b3ff1bac3cc26a34ef546907b68a64c3b9ca828bd37747c941505108"} Jun 12 18:38:07.948654 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:07.948264 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" event={"ID":"41fc3776e4bca81320bee88c10e1cb3f","Type":"ContainerStarted","Data":"ef36d1f53b9616d70249cefe1616b48dbce107df36b67dc8a5a24099e2a53ef5"} Jun 12 18:38:07.965326 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:07.965256 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-141.ec2.internal" podStartSLOduration=4.965234907 podStartE2EDuration="4.965234907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:07.964837819 +0000 UTC m=+5.531115055" watchObservedRunningTime="2026-06-12 18:38:07.965234907 +0000 UTC m=+5.531512148" Jun 12 18:38:08.478366 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.477777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:08.478366 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.477938 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:08.478366 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.478000 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:12.477982739 +0000 UTC m=+10.044259974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:08.578910 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.578870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:08.579087 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.578973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:08.579157 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579117 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:08.579212 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579179 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:12.579160265 +0000 UTC m=+10.145437494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:08.579282 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579261 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:08.579377 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579290 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:08.579377 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579304 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:08.579472 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.579378 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:12.579362045 +0000 UTC m=+10.145639267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:08.893944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.893909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:08.894134 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.894047 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:08.894480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.894458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:08.894604 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.894569 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:08.894691 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:08.894659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:08.894749 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:08.894727 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:10.893829 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:10.893796 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:10.893829 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:10.893819 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:10.894364 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:10.893944 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:10.894364 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:10.894001 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:10.894364 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:10.894052 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:10.894364 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:10.894135 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:12.509606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.509564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:12.510056 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.509719 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:12.510056 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.509795 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:20.509773814 +0000 UTC m=+18.076051058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:12.610163 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.610115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.610178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610307 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610341 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610355 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610368 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:12.610382 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610385 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:20.610365375 +0000 UTC m=+18.176642609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:12.610752 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.610404 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:20.610394391 +0000 UTC m=+18.176671605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.898837 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.899014 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.901297 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.901451 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:12.901542 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:12.903195 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:12.901654 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:14.897052 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:14.897090 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:14.897057 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:14.897180 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:14.897268 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:14.897450 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:14.897362 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:16.893695 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.893659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:16.893695 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.893680 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:16.894228 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:16.893790 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:16.894228 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.893833 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:16.894228 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:16.893926 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:16.894228 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:16.894014 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:16.969145 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.969108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerStarted","Data":"3046229779ccea67f1c46264680efcdf4480038646cf024bb3f6d13d08c7675d"} Jun 12 18:38:16.970772 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.970741 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z67bx" event={"ID":"9e5ce852-849e-4d39-bcbf-d6d44a3bdc36","Type":"ContainerStarted","Data":"16fe724cf383b4e13217aadbf09aaf175b04e220877d883a8787108ab0318ab3"} Jun 12 18:38:16.972398 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.972326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hc62t" event={"ID":"280a69e5-2a94-475f-9444-3b61d6fa80a9","Type":"ContainerStarted","Data":"f6d46a976a0b02c65736a118d4726e243b02558df431299d7af0569539925559"} Jun 12 18:38:16.973932 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.973876 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" event={"ID":"e888a720-68b1-47e8-b4e5-0bfb1e1454a6","Type":"ContainerStarted","Data":"7c9dc502548968d8b396f09759702858bd003ed5052bbc393eaab97a75111b7e"} Jun 12 18:38:16.975364 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.975341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rl5k" event={"ID":"b6661090-b2a0-4017-8029-bf18a56e6039","Type":"ContainerStarted","Data":"242d4cabfc37d9e1e7420874144e5b88d02541d121136af40f7a679278013f88"} Jun 12 18:38:16.976812 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:16.976791 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" event={"ID":"9a57803a-5b1f-4507-ba89-7059b1a7ce7e","Type":"ContainerStarted","Data":"20f795f00c6df0782072c0b57ffeed8aa772c24164d1e41a0f8a9df3395e02e7"} Jun 12 18:38:17.011949 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.011907 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8rl5k" podStartSLOduration=4.139358279 podStartE2EDuration="14.011895555s" totalImagesPullingTime="9.872537276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:17.011452971 +0000 UTC m=+14.577730203" watchObservedRunningTime="2026-06-12 18:38:17.011895555 +0000 UTC m=+14.578172791" Jun 12 18:38:17.066609 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.066401 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p4q5c" podStartSLOduration=3.831276224 podStartE2EDuration="14.066381844s" totalImagesPullingTime="10.23510562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:17.045617033 +0000 UTC m=+14.611894268" watchObservedRunningTime="2026-06-12 18:38:17.066381844 +0000 UTC m=+14.632659081" Jun 12 18:38:17.091403 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.091349 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hc62t" podStartSLOduration=3.8793498509999997 podStartE2EDuration="14.091334372s" totalImagesPullingTime="10.211984521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:17.066044925 +0000 UTC m=+14.632322190" watchObservedRunningTime="2026-06-12 18:38:17.091334372 +0000 UTC m=+14.657611608" Jun 12 18:38:17.980635 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.980583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6ljbd" event={"ID":"98db3c71-979c-4f56-ab78-577193af446f","Type":"ContainerStarted","Data":"d7cc28e47d957e06858c64866b39a3f104ea634e9f4508d7887ee85bc883fe9b"} Jun 12 18:38:17.995544 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.995491 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6ljbd" podStartSLOduration=5.127551306 podStartE2EDuration="14.995475193s" totalImagesPullingTime="9.867923887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:17.995202622 +0000 UTC m=+15.561479880" watchObservedRunningTime="2026-06-12 18:38:17.995475193 +0000 UTC m=+15.561752428" Jun 12 18:38:17.995914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:17.995881 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z67bx" podStartSLOduration=4.771602781 podStartE2EDuration="14.995871562s" totalImagesPullingTime="10.224268781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:17.091020258 +0000 UTC m=+14.657297494" watchObservedRunningTime="2026-06-12 18:38:17.995871562 +0000 UTC m=+15.562148799" Jun 12 18:38:18.894323 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:18.894289 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:18.894486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:18.894290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:18.894486 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:18.894406 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:18.894632 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:18.894509 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:18.894632 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:18.894301 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:18.894702 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:18.894633 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:19.145276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:19.145199 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:19.145798 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:19.145779 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:19.984340 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:19.984247 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:19.984869 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:19.984848 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8rl5k" Jun 12 18:38:20.570184 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.570153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:20.570861 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.570301 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:20.570861 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.570388 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:36.570366596 +0000 UTC m=+34.136643829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:20.670586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.670554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:20.670746 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.670609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:20.670746 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670702 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:20.670746 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670731 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:20.670889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670750 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:20.670889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670764 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:20.670889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670779 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:36.670759913 +0000 UTC m=+34.237037159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:20.670889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.670810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:38:36.670798468 +0000 UTC m=+34.237075681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:20.894412 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.894379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:20.894571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.894379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:20.894571 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.894513 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:20.894696 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.894603 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:20.894696 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.894379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:20.894796 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:20.894696 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:20.987098 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.987067 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="3046229779ccea67f1c46264680efcdf4480038646cf024bb3f6d13d08c7675d" exitCode=0 Jun 12 18:38:20.987259 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:20.987139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"3046229779ccea67f1c46264680efcdf4480038646cf024bb3f6d13d08c7675d"} Jun 12 18:38:22.894913 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:22.894824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:22.894913 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:22.894909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:22.895587 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:22.894954 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:22.895587 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:22.894963 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:22.895587 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:22.895052 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:22.895587 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:22.895116 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:24.899463 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.899225 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:24.899997 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:24.899525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:24.899997 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.899303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:24.899997 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:24.899621 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:24.899997 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.899257 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:24.899997 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:24.899667 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:24.940626 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.940590 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 12 18:38:24.996690 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.996661 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9hn6" event={"ID":"92c90488-bdc9-427f-b43c-e3dec7f332e2","Type":"ContainerStarted","Data":"d768a0f13b024286da688eccf48b0870df3e8f987e3bde7dc9e6922b2ed6a23c"} Jun 12 18:38:24.998619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:24.998591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" event={"ID":"e888a720-68b1-47e8-b4e5-0bfb1e1454a6","Type":"ContainerStarted","Data":"c287d98db97365f60d4530f6738b4d485a38d29b2aa0fa170e74dd2c3d903c90"} Jun 12 18:38:25.000786 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.000757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"d349db1fdf4a01aaaf61a46e39520ae69b645984245aba59f2cc0777c3a49474"} Jun 12 18:38:25.000900 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.000838 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"6718af17b4db1bcbe27b786ccc34b0295955f2d0f5f8cb28ca49ab7297e63071"} Jun 12 18:38:25.018003 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.017950 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s9hn6" podStartSLOduration=3.014987078 podStartE2EDuration="22.017930779s" totalImagesPullingTime="19.002943701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:25.017124262 +0000 UTC m=+22.583401530" watchObservedRunningTime="2026-06-12 18:38:25.017930779 +0000 UTC m=+22.584208019" Jun 12 18:38:25.911169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.910894 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-12T18:38:24.940613124Z","UUID":"99fd6aec-eb1e-4901-a447-52a02110584b","Handler":null,"Name":"","Endpoint":""} Jun 12 18:38:25.912877 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.912852 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 12 18:38:25.913015 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:25.912904 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 12 18:38:26.006642 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.006606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"7615c714e730b19d66f03c6a0925ddeccaf572d72239604f8461d1a56212705c"} Jun 12 18:38:26.006814 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.006686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"da1b3fb27bfe1ca4c9d01dd46f14b5e62b04d780ea7e3888d8e938a1a7a9556a"} Jun 12 18:38:26.006814 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.006726 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"b3b5565a6e975046b294be3456f4fcbc30ed9d1dcc4d3fb3604cdc77dae84342"} Jun 12 18:38:26.006814 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.006754 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"80a843b6142f9f3feac10ad9382ae4a8ff5e93c5b95f5f0b0c55d5d07c935aca"} Jun 12 18:38:26.897382 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.897352 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:26.897382 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.897389 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:26.897743 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:26.897387 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:26.897743 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:26.897470 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:26.897743 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:26.897578 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:26.897743 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:26.897707 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:27.011353 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:27.011301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" event={"ID":"e888a720-68b1-47e8-b4e5-0bfb1e1454a6","Type":"ContainerStarted","Data":"64323040bcd4851b6ba7c6333577336868acd5dc329ad2e890f4cb948bb02be4"} Jun 12 18:38:27.033287 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:27.033230 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8pxw" podStartSLOduration=4.031099285 podStartE2EDuration="24.033210943s" totalImagesPullingTime="20.002111658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:27.032910204 +0000 UTC m=+24.599187452" watchObservedRunningTime="2026-06-12 18:38:27.033210943 +0000 UTC m=+24.599488181" Jun 12 18:38:28.894373 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:28.894344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:28.894373 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:28.894358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:28.894892 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:28.894361 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:28.894892 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:28.894437 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:28.894892 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:28.894538 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:28.894892 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:28.894587 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:29.019328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:29.019291 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"f8fb0406f162e4f7072509e664c845bb51625cf5190dc796e9d617dbd9f28bea"} Jun 12 18:38:30.022523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:30.022495 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="12d9dad3a604e04db2d14885643cebaf86d30b088585d515341cd9984d7bbde1" exitCode=0 Jun 12 18:38:30.022855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:30.022529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"12d9dad3a604e04db2d14885643cebaf86d30b088585d515341cd9984d7bbde1"} Jun 12 18:38:30.896244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:30.896215 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:30.896507 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:30.896258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:30.896507 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:30.896333 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:30.896507 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:30.896357 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:30.896507 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:30.896408 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:30.896507 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:30.896460 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:32.029813 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.029575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" event={"ID":"b5f5bbfa-6d5d-4005-85f0-89303c4cc27e","Type":"ContainerStarted","Data":"60827b77b92a1ebfb31cf405002ea05e88288b97f606c1d428dad7e9545bc88c"} Jun 12 18:38:32.030488 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.029868 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:32.030488 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.029921 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:32.031856 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.031829 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="2c1914bf0d7d573b9ef3dd63b86a238475cc8b25cfe800042729a70fe1a92632" exitCode=0 Jun 12 18:38:32.032003 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.031861 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"2c1914bf0d7d573b9ef3dd63b86a238475cc8b25cfe800042729a70fe1a92632"} Jun 12 18:38:32.045087 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.045068 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:32.062475 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.062437 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" podStartSLOduration=10.064504021 podStartE2EDuration="29.062425494s" totalImagesPullingTime="18.997921473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:32.060719768 +0000 UTC m=+29.626997003" watchObservedRunningTime="2026-06-12 18:38:32.062425494 +0000 UTC m=+29.628702728" Jun 12 18:38:32.895211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.895180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:32.895486 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:32.895289 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:32.895486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.895305 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:32.895486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:32.895332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:32.895486 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:32.895424 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:32.895486 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:32.895477 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:33.035794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.035764 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:33.062230 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.062202 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:38:33.268515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.268484 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g7xz5"] Jun 12 18:38:33.268702 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.268584 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:33.268702 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:33.268670 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:33.269097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.269071 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkcrw"] Jun 12 18:38:33.269208 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.269153 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:33.269260 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:33.269235 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:33.271480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.271458 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8k97v"] Jun 12 18:38:33.271572 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:33.271537 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:33.271629 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:33.271611 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:34.038900 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:34.038868 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="870daa9b3c27ab4a74caa58661bcde2c39d14f50637723beae17d20fb8a48f2e" exitCode=0 Jun 12 18:38:34.039259 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:34.038944 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"870daa9b3c27ab4a74caa58661bcde2c39d14f50637723beae17d20fb8a48f2e"} Jun 12 18:38:34.894461 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:34.894244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:34.894638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:34.894245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:34.894638 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:34.894563 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:34.894638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:34.894281 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:34.894812 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:34.894687 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:34.894812 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:34.894731 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:36.595274 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.595239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:36.595692 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.595415 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:36.595692 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.595484 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret podName:4e14573a-c737-43c5-bd00-07679062a688 nodeName:}" failed. No retries permitted until 2026-06-12 18:39:08.595468255 +0000 UTC m=+66.161745484 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret") pod "global-pull-secret-syncer-dkcrw" (UID: "4e14573a-c737-43c5-bd00-07679062a688") : object "kube-system"/"original-pull-secret" not registered Jun 12 18:38:36.696023 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.695994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.696050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696131 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696149 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696167 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696177 2577 projected.go:196] Error preparing data for projected volume kube-api-access-hhz5n for pod openshift-network-diagnostics/network-check-target-g7xz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:36.696182 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696182 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:39:08.69616583 +0000 UTC m=+66.262443043 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 18:38:36.696396 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.696215 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n podName:34d03077-ed76-4e2f-a55b-dcdcdcf5401d nodeName:}" failed. No retries permitted until 2026-06-12 18:39:08.69620238 +0000 UTC m=+66.262479593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhz5n" (UniqueName: "kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n") pod "network-check-target-g7xz5" (UID: "34d03077-ed76-4e2f-a55b-dcdcdcf5401d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 18:38:36.894493 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.894464 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:36.894832 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.894506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:36.894832 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:36.894469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:36.894832 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.894604 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:38:36.894832 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.894675 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkcrw" podUID="4e14573a-c737-43c5-bd00-07679062a688" Jun 12 18:38:36.894832 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:36.894737 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g7xz5" podUID="34d03077-ed76-4e2f-a55b-dcdcdcf5401d" Jun 12 18:38:37.779378 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.779351 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-141.ec2.internal" event="NodeReady" Jun 12 18:38:37.779769 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.779483 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 12 18:38:37.836582 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.836550 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zjnct"] Jun 12 18:38:37.842204 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.842169 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:37.843062 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.843036 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-477dk"] Jun 12 18:38:37.844824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.844802 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwzlp\"" Jun 12 18:38:37.844919 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.844844 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 12 18:38:37.844919 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.844905 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 12 18:38:37.846288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.846271 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:37.848444 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.848426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 12 18:38:37.848986 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.848940 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 12 18:38:37.849131 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.849089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bw8dq\"" Jun 12 18:38:37.849282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.849265 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 12 18:38:37.853828 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.853793 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zjnct"] Jun 12 18:38:37.857582 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:37.857564 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-477dk"] Jun 12 18:38:38.004298 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.004298 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004297 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-tmp-dir\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.004542 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qhz4\" (UniqueName: \"kubernetes.io/projected/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-kube-api-access-7qhz4\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.004542 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-config-volume\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.004626 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpvg\" (UniqueName: \"kubernetes.io/projected/ffbd7381-8ee6-45e0-b64e-011e660db70a-kube-api-access-fwpvg\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.004626 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.004619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffbd7381-8ee6-45e0-b64e-011e660db70a-cert\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.105578 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105473 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffbd7381-8ee6-45e0-b64e-011e660db70a-cert\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.105578 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.105578 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-tmp-dir\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.105858 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qhz4\" (UniqueName: \"kubernetes.io/projected/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-kube-api-access-7qhz4\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.105858 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-config-volume\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.105858 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:38.105668 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:38.105858 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpvg\" (UniqueName: \"kubernetes.io/projected/ffbd7381-8ee6-45e0-b64e-011e660db70a-kube-api-access-fwpvg\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.105858 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:38.105740 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:38.605723824 +0000 UTC m=+36.172001037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:38:38.106112 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.105966 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-tmp-dir\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.106304 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.106283 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-config-volume\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.110457 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.110433 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffbd7381-8ee6-45e0-b64e-011e660db70a-cert\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.114607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.114585 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpvg\" (UniqueName: \"kubernetes.io/projected/ffbd7381-8ee6-45e0-b64e-011e660db70a-kube-api-access-fwpvg\") pod \"ingress-canary-477dk\" (UID: \"ffbd7381-8ee6-45e0-b64e-011e660db70a\") " pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.114944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.114924 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qhz4\" (UniqueName: \"kubernetes.io/projected/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-kube-api-access-7qhz4\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.161802 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.161771 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-477dk" Jun 12 18:38:38.574738 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.574712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-477dk"] Jun 12 18:38:38.577951 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:38:38.577912 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbd7381_8ee6_45e0_b64e_011e660db70a.slice/crio-e28397234f5eae9a7f0afc82bdcdb54ec6736bd81d65f4e331f56bed75cc4ea0 WatchSource:0}: Error finding container e28397234f5eae9a7f0afc82bdcdb54ec6736bd81d65f4e331f56bed75cc4ea0: Status 404 returned error can't find the container with id e28397234f5eae9a7f0afc82bdcdb54ec6736bd81d65f4e331f56bed75cc4ea0 Jun 12 18:38:38.609150 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.609125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:38.609293 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:38.609251 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:38.609454 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:38.609333 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:39.609297676 +0000 UTC m=+37.175574888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:38:38.893757 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.893721 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:38:38.894507 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.893724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:38:38.894507 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.893724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:38:38.896911 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.896886 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 12 18:38:38.897901 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.897887 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 12 18:38:38.898133 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.898119 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lhhs9\"" Jun 12 18:38:38.898250 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.898217 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 12 18:38:38.898250 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.898238 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 12 18:38:38.898431 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:38.898343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s5xp2\"" Jun 12 18:38:39.053240 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:39.053211 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="13755f90df099f392ab895ecdad5564fe8258d3ae2a5abb55ea7a05b90abee9f" exitCode=0 Jun 12 18:38:39.053417 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:39.053282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"13755f90df099f392ab895ecdad5564fe8258d3ae2a5abb55ea7a05b90abee9f"} Jun 12 18:38:39.054418 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:39.054393 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-477dk" event={"ID":"ffbd7381-8ee6-45e0-b64e-011e660db70a","Type":"ContainerStarted","Data":"e28397234f5eae9a7f0afc82bdcdb54ec6736bd81d65f4e331f56bed75cc4ea0"} Jun 12 18:38:39.616190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:39.616109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:39.616375 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:39.616286 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:39.616428 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:39.616394 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:41.616374468 +0000 UTC m=+39.182651702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:38:40.060039 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:40.060009 2577 generic.go:356] "Generic (PLEG): container finished" podID="96fd2d13-f5b8-4cc1-9192-a90e13ba3df1" containerID="555a7cec27f4b6bcf50657143aedd4ca76c6f44ec5ea4b027ea09dcd8a124eac" exitCode=0 Jun 12 18:38:40.060407 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:40.060048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerDied","Data":"555a7cec27f4b6bcf50657143aedd4ca76c6f44ec5ea4b027ea09dcd8a124eac"} Jun 12 18:38:41.065849 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:41.065821 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdh88" event={"ID":"96fd2d13-f5b8-4cc1-9192-a90e13ba3df1","Type":"ContainerStarted","Data":"c2382df170631725c8d48a96efe6d38db0a84781317ee9a34facf0384dfb55fb"} Jun 12 18:38:41.092187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:41.092145 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vdh88" podStartSLOduration=12.968352022 podStartE2EDuration="38.092130737s" totalImagesPullingTime="25.123778715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:41.090551644 +0000 UTC m=+38.656828881" watchObservedRunningTime="2026-06-12 18:38:41.092130737 +0000 UTC m=+38.658407971" Jun 12 18:38:41.629179 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:41.629147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:41.629373 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:41.629295 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:41.629430 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:41.629373 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:45.629357614 +0000 UTC m=+43.195634832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:38:42.069451 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:42.069423 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-477dk" event={"ID":"ffbd7381-8ee6-45e0-b64e-011e660db70a","Type":"ContainerStarted","Data":"a3cc799ecec670ec7ca059b89200bb0c3d01b440bbff0de1b5aa228f444caa97"} Jun 12 18:38:42.089267 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:42.089216 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-477dk" podStartSLOduration=2.654620373 podStartE2EDuration="5.089197107s" totalImagesPullingTime="2.434576734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:38:42.088390444 +0000 UTC m=+39.654667679" watchObservedRunningTime="2026-06-12 18:38:42.089197107 +0000 UTC m=+39.655474342" Jun 12 18:38:45.655099 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:45.654867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:45.655503 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:45.655011 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:45.655503 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:45.655235 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:38:53.655214166 +0000 UTC m=+51.221491379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:38:53.708126 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:38:53.708083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:38:53.708689 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:53.708215 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:38:53.708689 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:38:53.708274 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:39:09.708259327 +0000 UTC m=+67.274536541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:39:05.051341 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:05.051300 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdvng" Jun 12 18:39:08.606838 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.606798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:39:08.609931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.609910 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 12 18:39:08.621072 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.621047 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e14573a-c737-43c5-bd00-07679062a688-original-pull-secret\") pod \"global-pull-secret-syncer-dkcrw\" (UID: \"4e14573a-c737-43c5-bd00-07679062a688\") " pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:39:08.707495 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.707458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:39:08.707658 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.707521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:39:08.710255 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.710233 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 12 18:39:08.710533 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.710515 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 12 18:39:08.718444 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:08.718426 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 12 18:39:08.718512 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:08.718501 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:40:12.718479884 +0000 UTC m=+130.284757097 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : secret "metrics-daemon-secret" not found Jun 12 18:39:08.720717 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.720701 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 12 18:39:08.731134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.731103 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhz5n\" (UniqueName: \"kubernetes.io/projected/34d03077-ed76-4e2f-a55b-dcdcdcf5401d-kube-api-access-hhz5n\") pod \"network-check-target-g7xz5\" (UID: \"34d03077-ed76-4e2f-a55b-dcdcdcf5401d\") " pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:39:08.907202 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.907174 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s5xp2\"" Jun 12 18:39:08.914531 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.914513 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:39:08.914598 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:08.914590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkcrw" Jun 12 18:39:09.084238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.084204 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g7xz5"] Jun 12 18:39:09.087860 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:39:09.087825 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d03077_ed76_4e2f_a55b_dcdcdcf5401d.slice/crio-fb7ec41c86927cd7ed8aeefaaf586db20ad429c4d9608740c6f8f3f45944173d WatchSource:0}: Error finding container fb7ec41c86927cd7ed8aeefaaf586db20ad429c4d9608740c6f8f3f45944173d: Status 404 returned error can't find the container with id fb7ec41c86927cd7ed8aeefaaf586db20ad429c4d9608740c6f8f3f45944173d Jun 12 18:39:09.102574 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.102509 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkcrw"] Jun 12 18:39:09.105927 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:39:09.105893 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e14573a_c737_43c5_bd00_07679062a688.slice/crio-17bcb271b6ab23f91eb4eebb5deb90bae9e45d04a816fc7c7e83b800ec14a3a8 WatchSource:0}: Error finding container 17bcb271b6ab23f91eb4eebb5deb90bae9e45d04a816fc7c7e83b800ec14a3a8: Status 404 returned error can't find the container with id 17bcb271b6ab23f91eb4eebb5deb90bae9e45d04a816fc7c7e83b800ec14a3a8 Jun 12 18:39:09.133119 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.133084 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g7xz5" event={"ID":"34d03077-ed76-4e2f-a55b-dcdcdcf5401d","Type":"ContainerStarted","Data":"fb7ec41c86927cd7ed8aeefaaf586db20ad429c4d9608740c6f8f3f45944173d"} Jun 12 18:39:09.134180 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.134158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkcrw" event={"ID":"4e14573a-c737-43c5-bd00-07679062a688","Type":"ContainerStarted","Data":"17bcb271b6ab23f91eb4eebb5deb90bae9e45d04a816fc7c7e83b800ec14a3a8"} Jun 12 18:39:09.714345 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.714288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:39:09.714801 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:09.714466 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:39:09.714801 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:09.714552 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:39:41.714528838 +0000 UTC m=+99.280806067 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:39:09.799307 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.799272 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg"] Jun 12 18:39:09.820139 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.820084 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg"] Jun 12 18:39:09.820293 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.820203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:09.824515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.824488 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 12 18:39:09.824515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.824505 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 12 18:39:09.824515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.824508 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9t26q\"" Jun 12 18:39:09.824751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.824498 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 12 18:39:09.824836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.824817 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 12 18:39:09.916869 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.916740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d378065-19fa-4c8f-a8b5-a2beca02212e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:09.916869 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:09.916790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fjfb\" (UniqueName: \"kubernetes.io/projected/5d378065-19fa-4c8f-a8b5-a2beca02212e-kube-api-access-8fjfb\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.017504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.017406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d378065-19fa-4c8f-a8b5-a2beca02212e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.017504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.017466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fjfb\" (UniqueName: \"kubernetes.io/projected/5d378065-19fa-4c8f-a8b5-a2beca02212e-kube-api-access-8fjfb\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.020929 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.020901 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d378065-19fa-4c8f-a8b5-a2beca02212e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.027230 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.027176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fjfb\" (UniqueName: \"kubernetes.io/projected/5d378065-19fa-4c8f-a8b5-a2beca02212e-kube-api-access-8fjfb\") pod \"managed-serviceaccount-addon-agent-8695fc874-9wtdg\" (UID: \"5d378065-19fa-4c8f-a8b5-a2beca02212e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.152751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.152434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" Jun 12 18:39:10.317598 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:10.317495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg"] Jun 12 18:39:10.321874 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:39:10.321813 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d378065_19fa_4c8f_a8b5_a2beca02212e.slice/crio-e6740d9c6607c9c0f99d52ddf506e96040c0442598d2c34ead37ceb08bc48382 WatchSource:0}: Error finding container e6740d9c6607c9c0f99d52ddf506e96040c0442598d2c34ead37ceb08bc48382: Status 404 returned error can't find the container with id e6740d9c6607c9c0f99d52ddf506e96040c0442598d2c34ead37ceb08bc48382 Jun 12 18:39:11.142575 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:11.142535 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" event={"ID":"5d378065-19fa-4c8f-a8b5-a2beca02212e","Type":"ContainerStarted","Data":"e6740d9c6607c9c0f99d52ddf506e96040c0442598d2c34ead37ceb08bc48382"} Jun 12 18:39:15.155451 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.155415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkcrw" event={"ID":"4e14573a-c737-43c5-bd00-07679062a688","Type":"ContainerStarted","Data":"df464608cc3a56077ceb7533df2e7b96cb3d96bed4172373db913d958f3a16ee"} Jun 12 18:39:15.156834 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.156809 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g7xz5" event={"ID":"34d03077-ed76-4e2f-a55b-dcdcdcf5401d","Type":"ContainerStarted","Data":"58cf02265773b9f8c3a0472fe76cad27bae53331e5f7cfb7651311531a983740"} Jun 12 18:39:15.156950 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.156877 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:39:15.158203 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.158179 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" event={"ID":"5d378065-19fa-4c8f-a8b5-a2beca02212e","Type":"ContainerStarted","Data":"85d91df770d98a1f12296bd3779a445e70a1362e08e5ac4231d6c82832aa70f2"} Jun 12 18:39:15.182469 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.182427 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dkcrw" podStartSLOduration=66.422610244 podStartE2EDuration="1m12.182414128s" totalImagesPullingTime="5.759803884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:39:15.181236486 +0000 UTC m=+72.747513720" watchObservedRunningTime="2026-06-12 18:39:15.182414128 +0000 UTC m=+72.748691363" Jun 12 18:39:15.233491 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.233449 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g7xz5" podStartSLOduration=66.469899782 podStartE2EDuration="1m12.233435344s" totalImagesPullingTime="5.763535562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:39:15.212538709 +0000 UTC m=+72.778815968" watchObservedRunningTime="2026-06-12 18:39:15.233435344 +0000 UTC m=+72.799712556" Jun 12 18:39:15.233652 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:15.233633 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" podStartSLOduration=1.6948845970000002 podStartE2EDuration="6.233628009s" totalImagesPullingTime="4.538743412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:39:15.232897379 +0000 UTC m=+72.799174625" watchObservedRunningTime="2026-06-12 18:39:15.233628009 +0000 UTC m=+72.799905244" Jun 12 18:39:41.735806 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:41.735656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:39:41.736278 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:41.735817 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:39:41.736278 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:39:41.735887 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:40:45.735871309 +0000 UTC m=+163.302148522 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:39:46.163409 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:39:46.163373 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g7xz5" Jun 12 18:40:12.749976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:40:12.749915 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:40:12.750512 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:12.750090 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 12 18:40:12.750512 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:12.750160 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:14.75014406 +0000 UTC m=+252.316421277 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : secret "metrics-daemon-secret" not found Jun 12 18:40:40.854289 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:40.854228 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zjnct" podUID="5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9" Jun 12 18:40:41.361809 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:40:41.361774 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjnct" Jun 12 18:40:41.910165 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:41.910110 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:40:45.769259 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:40:45.769221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:40:45.769714 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:45.769383 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 18:40:45.769714 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:40:45.769455 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls podName:5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:47.76943793 +0000 UTC m=+285.335715161 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls") pod "dns-default-zjnct" (UID: "5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9") : secret "dns-default-metrics-tls" not found Jun 12 18:40:55.894211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:40:55.894129 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:41:15.442482 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:41:15.442446 2577 generic.go:356] "Generic (PLEG): container finished" podID="5d378065-19fa-4c8f-a8b5-a2beca02212e" containerID="85d91df770d98a1f12296bd3779a445e70a1362e08e5ac4231d6c82832aa70f2" exitCode=255 Jun 12 18:41:15.442857 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:41:15.442503 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" event={"ID":"5d378065-19fa-4c8f-a8b5-a2beca02212e","Type":"ContainerDied","Data":"85d91df770d98a1f12296bd3779a445e70a1362e08e5ac4231d6c82832aa70f2"} Jun 12 18:41:15.448111 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:41:15.448092 2577 scope.go:117] "RemoveContainer" containerID="85d91df770d98a1f12296bd3779a445e70a1362e08e5ac4231d6c82832aa70f2" Jun 12 18:41:16.446119 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:41:16.446083 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8695fc874-9wtdg" event={"ID":"5d378065-19fa-4c8f-a8b5-a2beca02212e","Type":"ContainerStarted","Data":"fb0c2b3079f6af093e9165e6fdd1259d0a6c39dd70adc45bc4be5f4c2e33d45d"} Jun 12 18:42:14.774746 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:14.774706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:42:14.775364 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:14.774878 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 12 18:42:14.775364 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:14.774970 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs podName:dd0084c5-7e69-4534-8301-20b771c71427 nodeName:}" failed. No retries permitted until 2026-06-12 18:44:16.774948009 +0000 UTC m=+374.341225221 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs") pod "network-metrics-daemon-8k97v" (UID: "dd0084c5-7e69-4534-8301-20b771c71427") : secret "metrics-daemon-secret" not found Jun 12 18:42:17.395840 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.395806 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4"] Jun 12 18:42:17.397832 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.397811 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.402210 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.402193 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dqhl5\"" Jun 12 18:42:17.403347 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.403330 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 12 18:42:17.403465 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.403448 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 12 18:42:17.404475 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.404458 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 12 18:42:17.406169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.406151 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 12 18:42:17.411971 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.411952 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-746857f878-zdttq"] Jun 12 18:42:17.413549 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.413536 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.418044 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.418009 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 12 18:42:17.418246 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.418107 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 12 18:42:17.418484 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.418461 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 12 18:42:17.418669 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.418649 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-82mlt\"" Jun 12 18:42:17.418669 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.418666 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 12 18:42:17.420721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.420700 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4"] Jun 12 18:42:17.423214 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.423199 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 12 18:42:17.424594 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.424581 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 12 18:42:17.437944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.437922 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc"] Jun 12 18:42:17.439679 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.439665 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" Jun 12 18:42:17.442444 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.442428 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4b74p\"" Jun 12 18:42:17.443182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.443168 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:42:17.443639 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.443628 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 12 18:42:17.445951 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.445929 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-746857f878-zdttq"] Jun 12 18:42:17.459729 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.459712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc"] Jun 12 18:42:17.481122 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.481096 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8"] Jun 12 18:42:17.482884 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.482868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" Jun 12 18:42:17.485867 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.485841 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9b5x4\"" Jun 12 18:42:17.491433 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.491408 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-8ngbd"] Jun 12 18:42:17.493230 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493216 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:42:17.493355 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.493818 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493801 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.493883 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhm7j\" (UniqueName: \"kubernetes.io/projected/019abd06-072a-4108-af28-ad66c55bbb36-kube-api-access-hhm7j\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.493883 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65tvc\" (UniqueName: \"kubernetes.io/projected/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-kube-api-access-65tvc\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.493972 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.493972 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493912 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.493972 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.494116 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.493996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-stats-auth\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.494116 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.494058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-default-certificate\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.496036 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.496019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.505073 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.505054 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 12 18:42:17.505168 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.505086 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 12 18:42:17.506468 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.506454 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8bl96\"" Jun 12 18:42:17.510369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.510353 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:42:17.514130 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.514110 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 12 18:42:17.515458 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.515439 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 12 18:42:17.516965 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.516946 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 12 18:42:17.518428 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.518408 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8"] Jun 12 18:42:17.527621 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.527600 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-8ngbd"] Jun 12 18:42:17.528128 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.528113 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ksm8k\"" Jun 12 18:42:17.529072 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.529051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 12 18:42:17.529287 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.529260 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 12 18:42:17.531958 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.531942 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 12 18:42:17.581535 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.581501 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:42:17.595387 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595356 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.595387 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-trusted-ca\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46rpd\" (UniqueName: \"kubernetes.io/projected/f3a8f43a-9208-4f97-b61b-451bdf1ef788-kube-api-access-46rpd\") pod \"network-check-source-6947749cfc-wtpb8\" (UID: \"f3a8f43a-9208-4f97-b61b-451bdf1ef788\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-stats-auth\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-default-certificate\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.595602 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.595546 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.595697 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:18.095675074 +0000 UTC m=+255.661952288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kks4v\" (UniqueName: \"kubernetes.io/projected/987a55ab-eb39-48f6-886d-780f76778c0c-kube-api-access-kks4v\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhm7j\" (UniqueName: \"kubernetes.io/projected/019abd06-072a-4108-af28-ad66c55bbb36-kube-api-access-hhm7j\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595788 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65tvc\" (UniqueName: \"kubernetes.io/projected/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-kube-api-access-65tvc\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-config\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595933 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/987a55ab-eb39-48f6-886d-780f76778c0c-serving-cert\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.595975 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 12 18:42:17.595981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.595984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.596029 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:18.096012095 +0000 UTC m=+255.662289308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : secret "router-metrics-certs-default" not found Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.596049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvm2p\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.596057 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:18.096043688 +0000 UTC m=+255.662320907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.596093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gws4\" (UniqueName: \"kubernetes.io/projected/711b930d-d2b7-4af5-be1d-a8dc2550462f-kube-api-access-8gws4\") pod \"volume-data-source-validator-bd4ccd799-9dnkc\" (UID: \"711b930d-d2b7-4af5-be1d-a8dc2550462f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.596119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.596504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.596363 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.598064 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.598048 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-default-certificate\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.598205 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.598187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-stats-auth\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.623998 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.623968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65tvc\" (UniqueName: \"kubernetes.io/projected/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-kube-api-access-65tvc\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:17.627556 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.627525 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhm7j\" (UniqueName: \"kubernetes.io/projected/019abd06-072a-4108-af28-ad66c55bbb36-kube-api-access-hhm7j\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:17.696607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gws4\" (UniqueName: \"kubernetes.io/projected/711b930d-d2b7-4af5-be1d-a8dc2550462f-kube-api-access-8gws4\") pod \"volume-data-source-validator-bd4ccd799-9dnkc\" (UID: \"711b930d-d2b7-4af5-be1d-a8dc2550462f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" Jun 12 18:42:17.696607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.696607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-trusted-ca\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.696895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46rpd\" (UniqueName: \"kubernetes.io/projected/f3a8f43a-9208-4f97-b61b-451bdf1ef788-kube-api-access-46rpd\") pod \"network-check-source-6947749cfc-wtpb8\" (UID: \"f3a8f43a-9208-4f97-b61b-451bdf1ef788\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" Jun 12 18:42:17.696895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.696895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kks4v\" (UniqueName: \"kubernetes.io/projected/987a55ab-eb39-48f6-886d-780f76778c0c-kube-api-access-kks4v\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.697052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.696979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-config\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.697052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697037 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697043 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/987a55ab-eb39-48f6-886d-780f76778c0c-serving-cert\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.697126 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.697142 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676759d85f-5xw5b: secret "image-registry-tls" not found Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvm2p\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697370 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:17.697206 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls podName:f2eaf655-5707-4380-996e-15531eecd4a3 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:18.197189241 +0000 UTC m=+255.763466466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls") pod "image-registry-676759d85f-5xw5b" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3") : secret "image-registry-tls" not found Jun 12 18:42:17.697752 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697491 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.697752 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697712 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-config\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.697752 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697734 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/987a55ab-eb39-48f6-886d-780f76778c0c-trusted-ca\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.697951 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.697927 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.699738 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.699710 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/987a55ab-eb39-48f6-886d-780f76778c0c-serving-cert\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.699907 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.699834 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.699907 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.699900 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.706424 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.706398 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46rpd\" (UniqueName: \"kubernetes.io/projected/f3a8f43a-9208-4f97-b61b-451bdf1ef788-kube-api-access-46rpd\") pod \"network-check-source-6947749cfc-wtpb8\" (UID: \"f3a8f43a-9208-4f97-b61b-451bdf1ef788\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" Jun 12 18:42:17.706535 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.706485 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.706651 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.706634 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kks4v\" (UniqueName: \"kubernetes.io/projected/987a55ab-eb39-48f6-886d-780f76778c0c-kube-api-access-kks4v\") pod \"console-operator-5dbbb5d744-8ngbd\" (UID: \"987a55ab-eb39-48f6-886d-780f76778c0c\") " pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.707035 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.707019 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gws4\" (UniqueName: \"kubernetes.io/projected/711b930d-d2b7-4af5-be1d-a8dc2550462f-kube-api-access-8gws4\") pod \"volume-data-source-validator-bd4ccd799-9dnkc\" (UID: \"711b930d-d2b7-4af5-be1d-a8dc2550462f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" Jun 12 18:42:17.709738 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.709722 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvm2p\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:17.748839 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.748814 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" Jun 12 18:42:17.791680 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.791642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" Jun 12 18:42:17.803297 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.803268 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:17.886407 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.886374 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc"] Jun 12 18:42:17.890897 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:17.890856 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod711b930d_d2b7_4af5_be1d_a8dc2550462f.slice/crio-c4bc593b07e09c3b084ae948375faa18a5aae0ce435627fe321befb55cf406e8 WatchSource:0}: Error finding container c4bc593b07e09c3b084ae948375faa18a5aae0ce435627fe321befb55cf406e8: Status 404 returned error can't find the container with id c4bc593b07e09c3b084ae948375faa18a5aae0ce435627fe321befb55cf406e8 Jun 12 18:42:17.950443 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.950372 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8"] Jun 12 18:42:17.953409 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:17.953381 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a8f43a_9208_4f97_b61b_451bdf1ef788.slice/crio-2891cb946808aa183aa2545ca9f10e15e0f1aed4c1604a2efd23b910c96b230c WatchSource:0}: Error finding container 2891cb946808aa183aa2545ca9f10e15e0f1aed4c1604a2efd23b910c96b230c: Status 404 returned error can't find the container with id 2891cb946808aa183aa2545ca9f10e15e0f1aed4c1604a2efd23b910c96b230c Jun 12 18:42:17.956831 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:17.956810 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-8ngbd"] Jun 12 18:42:17.959947 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:17.959926 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987a55ab_eb39_48f6_886d_780f76778c0c.slice/crio-6213cb957daafaddf3579ef40a6e47dd9324e9221263e1029d27f7e0002deef4 WatchSource:0}: Error finding container 6213cb957daafaddf3579ef40a6e47dd9324e9221263e1029d27f7e0002deef4: Status 404 returned error can't find the container with id 6213cb957daafaddf3579ef40a6e47dd9324e9221263e1029d27f7e0002deef4 Jun 12 18:42:18.100681 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.100639 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:18.100889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.100794 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:18.100889 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.100814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:18.100889 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.100875 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:19.10085595 +0000 UTC m=+256.667133184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:18.100999 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.100896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:18.100999 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.100913 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 12 18:42:18.100999 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.100950 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:19.100938582 +0000 UTC m=+256.667215802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : secret "router-metrics-certs-default" not found Jun 12 18:42:18.101092 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.101003 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:19.100996 +0000 UTC m=+256.667273212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:18.201670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.201571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:18.201845 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.201743 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 18:42:18.201845 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.201763 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676759d85f-5xw5b: secret "image-registry-tls" not found Jun 12 18:42:18.201845 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:18.201820 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls podName:f2eaf655-5707-4380-996e-15531eecd4a3 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:19.201800121 +0000 UTC m=+256.768077334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls") pod "image-registry-676759d85f-5xw5b" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3") : secret "image-registry-tls" not found Jun 12 18:42:18.592778 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.592663 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" event={"ID":"987a55ab-eb39-48f6-886d-780f76778c0c","Type":"ContainerStarted","Data":"6213cb957daafaddf3579ef40a6e47dd9324e9221263e1029d27f7e0002deef4"} Jun 12 18:42:18.594276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.594246 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" event={"ID":"711b930d-d2b7-4af5-be1d-a8dc2550462f","Type":"ContainerStarted","Data":"c4bc593b07e09c3b084ae948375faa18a5aae0ce435627fe321befb55cf406e8"} Jun 12 18:42:18.596061 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.596034 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" event={"ID":"f3a8f43a-9208-4f97-b61b-451bdf1ef788","Type":"ContainerStarted","Data":"8ad09a0decd99db786114ac6237b237e4fcf2195a7658ee40fe081894bd8e186"} Jun 12 18:42:18.596153 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.596084 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" event={"ID":"f3a8f43a-9208-4f97-b61b-451bdf1ef788","Type":"ContainerStarted","Data":"2891cb946808aa183aa2545ca9f10e15e0f1aed4c1604a2efd23b910c96b230c"} Jun 12 18:42:18.627897 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:18.627839 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6947749cfc-wtpb8" podStartSLOduration=1.627819269 podStartE2EDuration="1.627819269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:18.626990482 +0000 UTC m=+256.193267718" watchObservedRunningTime="2026-06-12 18:42:18.627819269 +0000 UTC m=+256.194096504" Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.110204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.110301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.110363 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.110385 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.110473 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:21.110435558 +0000 UTC m=+258.676712777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.110514 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:21.110499119 +0000 UTC m=+258.676776346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.110587 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 12 18:42:19.110648 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.110625 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:21.110613545 +0000 UTC m=+258.676890771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : secret "router-metrics-certs-default" not found Jun 12 18:42:19.211591 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.211550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:19.211768 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.211723 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 18:42:19.211768 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.211748 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676759d85f-5xw5b: secret "image-registry-tls" not found Jun 12 18:42:19.211868 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:19.211818 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls podName:f2eaf655-5707-4380-996e-15531eecd4a3 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:21.211796913 +0000 UTC m=+258.778074133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls") pod "image-registry-676759d85f-5xw5b" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3") : secret "image-registry-tls" not found Jun 12 18:42:19.600740 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.600698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" event={"ID":"711b930d-d2b7-4af5-be1d-a8dc2550462f","Type":"ContainerStarted","Data":"eb9e39504609f17a1a7ace0a93f82a239a3e97f98d98f30dcc14fe611646c6a2"} Jun 12 18:42:19.618967 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:19.618924 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-9dnkc" podStartSLOduration=1.483105884 podStartE2EDuration="2.618908894s" totalImagesPullingTime="1.13580301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:19.618082084 +0000 UTC m=+257.184359317" watchObservedRunningTime="2026-06-12 18:42:19.618908894 +0000 UTC m=+257.185186128" Jun 12 18:42:20.605416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:20.605386 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/0.log" Jun 12 18:42:20.605894 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:20.605426 2577 generic.go:356] "Generic (PLEG): container finished" podID="987a55ab-eb39-48f6-886d-780f76778c0c" containerID="4b93e3a1123dde5b2287480fdc856f3ab111f6b5f7ae43f868575d2c52490423" exitCode=255 Jun 12 18:42:20.605894 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:20.605499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" event={"ID":"987a55ab-eb39-48f6-886d-780f76778c0c","Type":"ContainerDied","Data":"4b93e3a1123dde5b2287480fdc856f3ab111f6b5f7ae43f868575d2c52490423"} Jun 12 18:42:20.605894 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:20.605681 2577 scope.go:117] "RemoveContainer" containerID="4b93e3a1123dde5b2287480fdc856f3ab111f6b5f7ae43f868575d2c52490423" Jun 12 18:42:21.128609 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.128565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:21.128805 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.128626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:21.128805 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.128736 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:21.128805 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.128747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:21.128970 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.128822 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:25.128802675 +0000 UTC m=+262.695079889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:21.128970 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.128843 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 12 18:42:21.128970 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.128873 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:25.12885592 +0000 UTC m=+262.695133133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:21.128970 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.128896 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:25.12888489 +0000 UTC m=+262.695162108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : secret "router-metrics-certs-default" not found Jun 12 18:42:21.229136 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.229103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:21.229303 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.229202 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 18:42:21.229303 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.229212 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676759d85f-5xw5b: secret "image-registry-tls" not found Jun 12 18:42:21.229303 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.229256 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls podName:f2eaf655-5707-4380-996e-15531eecd4a3 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:25.229244543 +0000 UTC m=+262.795521755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls") pod "image-registry-676759d85f-5xw5b" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3") : secret "image-registry-tls" not found Jun 12 18:42:21.610238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610167 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:42:21.610670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610525 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/0.log" Jun 12 18:42:21.610670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610556 2577 generic.go:356] "Generic (PLEG): container finished" podID="987a55ab-eb39-48f6-886d-780f76778c0c" containerID="d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9" exitCode=255 Jun 12 18:42:21.610670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" event={"ID":"987a55ab-eb39-48f6-886d-780f76778c0c","Type":"ContainerDied","Data":"d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9"} Jun 12 18:42:21.610670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610623 2577 scope.go:117] "RemoveContainer" containerID="4b93e3a1123dde5b2287480fdc856f3ab111f6b5f7ae43f868575d2c52490423" Jun 12 18:42:21.610913 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:21.610892 2577 scope.go:117] "RemoveContainer" containerID="d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9" Jun 12 18:42:21.611067 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:21.611050 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-8ngbd_openshift-console-operator(987a55ab-eb39-48f6-886d-780f76778c0c)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" podUID="987a55ab-eb39-48f6-886d-780f76778c0c" Jun 12 18:42:22.303826 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.303794 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qm4m2"] Jun 12 18:42:22.306525 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.306507 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.312807 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.312788 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 12 18:42:22.312807 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.312803 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 12 18:42:22.315323 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.315298 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t6vkb\"" Jun 12 18:42:22.315610 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.315598 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 12 18:42:22.315670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.315647 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 12 18:42:22.324557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.324537 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qm4m2"] Jun 12 18:42:22.438103 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.438066 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e1989e3-9399-457d-99be-b16e05f27d91-data-volume\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.438265 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.438206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgdx\" (UniqueName: \"kubernetes.io/projected/2e1989e3-9399-457d-99be-b16e05f27d91-kube-api-access-mtgdx\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.438265 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.438257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.438409 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.438291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e1989e3-9399-457d-99be-b16e05f27d91-crio-socket\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.438409 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.438363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539525 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539494 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgdx\" (UniqueName: \"kubernetes.io/projected/2e1989e3-9399-457d-99be-b16e05f27d91-kube-api-access-mtgdx\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539525 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e1989e3-9399-457d-99be-b16e05f27d91-crio-socket\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:22.539633 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:22.539649 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539649 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e1989e3-9399-457d-99be-b16e05f27d91-crio-socket\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.539729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e1989e3-9399-457d-99be-b16e05f27d91-data-volume\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.539989 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:22.539756 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:23.039740016 +0000 UTC m=+260.606017232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : secret "insights-runtime-extractor-tls" not found Jun 12 18:42:22.539989 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:22.539782 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:23.039774822 +0000 UTC m=+260.606052035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : configmap "kube-rbac-proxy" not found Jun 12 18:42:22.540068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.540023 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e1989e3-9399-457d-99be-b16e05f27d91-data-volume\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.547853 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.547830 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgdx\" (UniqueName: \"kubernetes.io/projected/2e1989e3-9399-457d-99be-b16e05f27d91-kube-api-access-mtgdx\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:22.614713 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.614635 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:42:22.615112 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:22.614924 2577 scope.go:117] "RemoveContainer" containerID="d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9" Jun 12 18:42:22.615112 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:22.615087 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-8ngbd_openshift-console-operator(987a55ab-eb39-48f6-886d-780f76778c0c)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" podUID="987a55ab-eb39-48f6-886d-780f76778c0c" Jun 12 18:42:23.044082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:23.044049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:23.044082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:23.044086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:23.044274 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:23.044165 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 18:42:23.044274 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:23.044191 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 18:42:23.044274 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:23.044233 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:24.044220835 +0000 UTC m=+261.610498051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : secret "insights-runtime-extractor-tls" not found Jun 12 18:42:23.044274 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:23.044245 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:24.044239163 +0000 UTC m=+261.610516376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : configmap "kube-rbac-proxy" not found Jun 12 18:42:24.051212 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:24.051173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:24.051212 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:24.051214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:24.051654 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:24.051294 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 18:42:24.051654 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:24.051360 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 18:42:24.051654 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:24.051383 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:26.051364799 +0000 UTC m=+263.617642037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : configmap "kube-rbac-proxy" not found Jun 12 18:42:24.051654 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:24.051398 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:26.051391744 +0000 UTC m=+263.617668957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : secret "insights-runtime-extractor-tls" not found Jun 12 18:42:24.267920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:24.267893 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hc62t_280a69e5-2a94-475f-9444-3b61d6fa80a9/dns-node-resolver/0.log" Jun 12 18:42:25.161993 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.161946 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.162016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.162055 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.162118 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:33.16210252 +0000 UTC m=+270.728379735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : secret "router-metrics-certs-default" not found Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.162153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.162214 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:33.162193838 +0000 UTC m=+270.728471055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.162231 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:25.162455 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.162257 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:33.162249928 +0000 UTC m=+270.728527140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:25.263109 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.263071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:25.263271 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.263252 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 18:42:25.263336 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.263274 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676759d85f-5xw5b: secret "image-registry-tls" not found Jun 12 18:42:25.263426 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:25.263350 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls podName:f2eaf655-5707-4380-996e-15531eecd4a3 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:33.263331465 +0000 UTC m=+270.829608680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls") pod "image-registry-676759d85f-5xw5b" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3") : secret "image-registry-tls" not found Jun 12 18:42:25.466791 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.466714 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z67bx_9e5ce852-849e-4d39-bcbf-d6d44a3bdc36/node-ca/0.log" Jun 12 18:42:25.875552 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:25.875469 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-477dk_ffbd7381-8ee6-45e0-b64e-011e660db70a/serve-healthcheck-canary/0.log" Jun 12 18:42:26.069467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:26.069421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:26.069467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:26.069472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:26.069707 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:26.069545 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 18:42:26.069707 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:26.069561 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 18:42:26.069707 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:26.069621 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:30.069608709 +0000 UTC m=+267.635885921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : secret "insights-runtime-extractor-tls" not found Jun 12 18:42:26.069707 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:26.069694 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:30.069674274 +0000 UTC m=+267.635951490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : configmap "kube-rbac-proxy" not found Jun 12 18:42:27.803941 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:27.803853 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:27.803941 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:27.803928 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:27.804368 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:27.804248 2577 scope.go:117] "RemoveContainer" containerID="d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9" Jun 12 18:42:27.804430 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:27.804411 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-8ngbd_openshift-console-operator(987a55ab-eb39-48f6-886d-780f76778c0c)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" podUID="987a55ab-eb39-48f6-886d-780f76778c0c" Jun 12 18:42:30.102502 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:30.102461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:30.102932 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:30.102605 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 18:42:30.102932 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:30.102622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:30.102932 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:30.102667 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:38.102650377 +0000 UTC m=+275.668927590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : secret "insights-runtime-extractor-tls" not found Jun 12 18:42:30.102932 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:30.102710 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 18:42:30.102932 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:30.102766 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm podName:2e1989e3-9399-457d-99be-b16e05f27d91 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:38.102750712 +0000 UTC m=+275.669027927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qm4m2" (UID: "2e1989e3-9399-457d-99be-b16e05f27d91") : configmap "kube-rbac-proxy" not found Jun 12 18:42:33.232279 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.232238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:33.232717 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.232329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:33.232717 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.232368 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:33.232717 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:33.232399 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:33.232717 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:33.232466 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls podName:aff6bbf7-a1a9-42b3-aa6f-36331e58d629 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:49.232451665 +0000 UTC m=+286.798728882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-d4vg4" (UID: "aff6bbf7-a1a9-42b3-aa6f-36331e58d629") : secret "cluster-monitoring-operator-tls" not found Jun 12 18:42:33.232717 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:33.232491 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle podName:019abd06-072a-4108-af28-ad66c55bbb36 nodeName:}" failed. No retries permitted until 2026-06-12 18:42:49.232476253 +0000 UTC m=+286.798753465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle") pod "router-default-746857f878-zdttq" (UID: "019abd06-072a-4108-af28-ad66c55bbb36") : configmap references non-existent config key: service-ca.crt Jun 12 18:42:33.234746 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.234727 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019abd06-072a-4108-af28-ad66c55bbb36-metrics-certs\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:33.333740 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.333697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:33.336129 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.336101 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"image-registry-676759d85f-5xw5b\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:33.407067 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.407043 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:33.539482 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.539456 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:42:33.541728 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:33.541704 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2eaf655_5707_4380_996e_15531eecd4a3.slice/crio-ca13e832b4c86e4083b2acd132cea4986c528497271e585a674228fe5c4e864b WatchSource:0}: Error finding container ca13e832b4c86e4083b2acd132cea4986c528497271e585a674228fe5c4e864b: Status 404 returned error can't find the container with id ca13e832b4c86e4083b2acd132cea4986c528497271e585a674228fe5c4e864b Jun 12 18:42:33.648675 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.648638 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" event={"ID":"f2eaf655-5707-4380-996e-15531eecd4a3","Type":"ContainerStarted","Data":"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97"} Jun 12 18:42:33.648848 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.648700 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" event={"ID":"f2eaf655-5707-4380-996e-15531eecd4a3","Type":"ContainerStarted","Data":"ca13e832b4c86e4083b2acd132cea4986c528497271e585a674228fe5c4e864b"} Jun 12 18:42:33.648848 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.648776 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:42:33.670108 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:33.670051 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" podStartSLOduration=16.670030634 podStartE2EDuration="16.670030634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:33.668684162 +0000 UTC m=+271.234961398" watchObservedRunningTime="2026-06-12 18:42:33.670030634 +0000 UTC m=+271.236307869" Jun 12 18:42:38.175024 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.174983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:38.175024 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.175024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:38.175531 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.175510 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e1989e3-9399-457d-99be-b16e05f27d91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:38.177272 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.177245 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e1989e3-9399-457d-99be-b16e05f27d91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qm4m2\" (UID: \"2e1989e3-9399-457d-99be-b16e05f27d91\") " pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:38.215440 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.215410 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qm4m2" Jun 12 18:42:38.347824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.347790 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qm4m2"] Jun 12 18:42:38.350500 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:38.350470 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1989e3_9399_457d_99be_b16e05f27d91.slice/crio-13d1a574fc996402e97ecd9a9af30362a1a224dc607a70ba06eb08cb7dd2188d WatchSource:0}: Error finding container 13d1a574fc996402e97ecd9a9af30362a1a224dc607a70ba06eb08cb7dd2188d: Status 404 returned error can't find the container with id 13d1a574fc996402e97ecd9a9af30362a1a224dc607a70ba06eb08cb7dd2188d Jun 12 18:42:38.664985 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:38.664889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qm4m2" event={"ID":"2e1989e3-9399-457d-99be-b16e05f27d91","Type":"ContainerStarted","Data":"13d1a574fc996402e97ecd9a9af30362a1a224dc607a70ba06eb08cb7dd2188d"} Jun 12 18:42:40.894803 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:40.894767 2577 scope.go:117] "RemoveContainer" containerID="d824f01a076947c2526a20e09293126e3ef1446d8dcb7d7869704b2536fdd6c9" Jun 12 18:42:41.679093 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.679056 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qm4m2" event={"ID":"2e1989e3-9399-457d-99be-b16e05f27d91","Type":"ContainerStarted","Data":"68cf87a40e26b7572bd78a22a287ed33894f9b4740d147e0bba63f501ea3aece"} Jun 12 18:42:41.680986 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.680967 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:42:41.681079 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.681017 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" event={"ID":"987a55ab-eb39-48f6-886d-780f76778c0c","Type":"ContainerStarted","Data":"31da4a675e5eb3be7115324938c67b5b5238b5884ab27b8bda91264cc4b0de94"} Jun 12 18:42:41.681300 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.681281 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:41.699728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.699684 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" podStartSLOduration=22.862523218 podStartE2EDuration="24.699667015s" totalImagesPullingTime="1.837143797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:41.698824127 +0000 UTC m=+279.265101375" watchObservedRunningTime="2026-06-12 18:42:41.699667015 +0000 UTC m=+279.265944285" Jun 12 18:42:41.996440 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:41.996413 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5dbbb5d744-8ngbd" Jun 12 18:42:42.687244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:42.687204 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qm4m2" event={"ID":"2e1989e3-9399-457d-99be-b16e05f27d91","Type":"ContainerStarted","Data":"f5d376884c9bc0050fcb02ff76e4b40f07a99ba68073be498920f242284a6c4d"} Jun 12 18:42:44.362466 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:44.362421 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zjnct" podUID="5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9" Jun 12 18:42:44.695638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:44.695613 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjnct" Jun 12 18:42:44.695638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:44.695628 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qm4m2" event={"ID":"2e1989e3-9399-457d-99be-b16e05f27d91","Type":"ContainerStarted","Data":"80f8c3cc1d66e7773919b7b833a6826820ff8011a46e7d4721b79479417b9831"} Jun 12 18:42:44.717600 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:44.717549 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qm4m2" podStartSLOduration=17.40718097 podStartE2EDuration="22.717533524s" totalImagesPullingTime="5.310352554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 18:42:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:44.716472523 +0000 UTC m=+282.282749758" watchObservedRunningTime="2026-06-12 18:42:44.717533524 +0000 UTC m=+282.283810759" Jun 12 18:42:47.858344 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:47.858287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:42:47.860559 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:47.860539 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9-metrics-tls\") pod \"dns-default-zjnct\" (UID: \"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9\") " pod="openshift-dns/dns-default-zjnct" Jun 12 18:42:47.998840 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:47.998807 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwzlp\"" Jun 12 18:42:48.007105 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:48.007083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjnct" Jun 12 18:42:48.132938 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:48.132910 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zjnct"] Jun 12 18:42:48.135397 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:48.135365 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2799d1_8e1e_4572_9ca2_c3453f3ec3b9.slice/crio-183004c072e1c060086a65bbe713d8d8d349fc4f0cd2dd56e9b523b79b93be9b WatchSource:0}: Error finding container 183004c072e1c060086a65bbe713d8d8d349fc4f0cd2dd56e9b523b79b93be9b: Status 404 returned error can't find the container with id 183004c072e1c060086a65bbe713d8d8d349fc4f0cd2dd56e9b523b79b93be9b Jun 12 18:42:48.708758 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:48.708724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjnct" event={"ID":"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9","Type":"ContainerStarted","Data":"183004c072e1c060086a65bbe713d8d8d349fc4f0cd2dd56e9b523b79b93be9b"} Jun 12 18:42:49.269423 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.269384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:49.269837 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.269462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:49.270142 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.270120 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019abd06-072a-4108-af28-ad66c55bbb36-service-ca-bundle\") pod \"router-default-746857f878-zdttq\" (UID: \"019abd06-072a-4108-af28-ad66c55bbb36\") " pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:49.272063 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.272034 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aff6bbf7-a1a9-42b3-aa6f-36331e58d629-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-d4vg4\" (UID: \"aff6bbf7-a1a9-42b3-aa6f-36331e58d629\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:49.370115 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.370081 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:42:49.374552 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.374521 2577 patch_prober.go:28] interesting pod/image-registry-676759d85f-5xw5b container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 12 18:42:49.374702 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.374598 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 12 18:42:49.506570 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.506532 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" Jun 12 18:42:49.522050 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.521990 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:49.664281 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.664251 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4"] Jun 12 18:42:49.668142 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:49.668113 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff6bbf7_a1a9_42b3_aa6f_36331e58d629.slice/crio-07923b23203364701eac972ce9cb3be46ad9e67a9bfdaaa5a9a28b0e9fe79eab WatchSource:0}: Error finding container 07923b23203364701eac972ce9cb3be46ad9e67a9bfdaaa5a9a28b0e9fe79eab: Status 404 returned error can't find the container with id 07923b23203364701eac972ce9cb3be46ad9e67a9bfdaaa5a9a28b0e9fe79eab Jun 12 18:42:49.693601 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.693565 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-746857f878-zdttq"] Jun 12 18:42:49.696732 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:49.696707 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019abd06_072a_4108_af28_ad66c55bbb36.slice/crio-2d69797c634019291b4d044a43f0ddb6fe2a3b0bdf60e93a821c4c0592e0c17d WatchSource:0}: Error finding container 2d69797c634019291b4d044a43f0ddb6fe2a3b0bdf60e93a821c4c0592e0c17d: Status 404 returned error can't find the container with id 2d69797c634019291b4d044a43f0ddb6fe2a3b0bdf60e93a821c4c0592e0c17d Jun 12 18:42:49.713861 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.713837 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-746857f878-zdttq" event={"ID":"019abd06-072a-4108-af28-ad66c55bbb36","Type":"ContainerStarted","Data":"2d69797c634019291b4d044a43f0ddb6fe2a3b0bdf60e93a821c4c0592e0c17d"} Jun 12 18:42:49.714873 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:49.714849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" event={"ID":"aff6bbf7-a1a9-42b3-aa6f-36331e58d629","Type":"ContainerStarted","Data":"07923b23203364701eac972ce9cb3be46ad9e67a9bfdaaa5a9a28b0e9fe79eab"} Jun 12 18:42:50.721302 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:50.721270 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-746857f878-zdttq" event={"ID":"019abd06-072a-4108-af28-ad66c55bbb36","Type":"ContainerStarted","Data":"34b3a4cba1a482571e40ae1d6b5c0bb6da1cf4b36b79434605bec6c8de12e392"} Jun 12 18:42:50.722771 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:50.722745 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjnct" event={"ID":"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9","Type":"ContainerStarted","Data":"2720e88a2fe0b8fea82b8966a071b8a1165551e8b32f8bae9298e32dd69ff3da"} Jun 12 18:42:50.742210 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:50.742153 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-746857f878-zdttq" podStartSLOduration=33.742131318 podStartE2EDuration="33.742131318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:50.741146935 +0000 UTC m=+288.307424171" watchObservedRunningTime="2026-06-12 18:42:50.742131318 +0000 UTC m=+288.308408557" Jun 12 18:42:51.522170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.522145 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:51.524570 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.524550 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:51.728140 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.728055 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" event={"ID":"aff6bbf7-a1a9-42b3-aa6f-36331e58d629","Type":"ContainerStarted","Data":"8352ae626eb3e3bbc1635f3f48221a2fdd293f42fcac70db64dfdbc07fd91790"} Jun 12 18:42:51.729920 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.729891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjnct" event={"ID":"5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9","Type":"ContainerStarted","Data":"9d44e8a9a999e8140b63acda2213fa29ea9804cad7d1fe0da34eb1027f3900a4"} Jun 12 18:42:51.730023 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.729995 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:51.730066 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.730048 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zjnct" Jun 12 18:42:51.731348 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.731327 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-746857f878-zdttq" Jun 12 18:42:51.754302 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.754264 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-d4vg4" podStartSLOduration=32.964918739 podStartE2EDuration="34.754250602s" totalImagesPullingTime="1.789331863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:51.753672637 +0000 UTC m=+289.319949874" watchObservedRunningTime="2026-06-12 18:42:51.754250602 +0000 UTC m=+289.320527881" Jun 12 18:42:51.797324 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:51.797251 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zjnct" podStartSLOduration=252.371234129 podStartE2EDuration="4m14.797231727s" totalImagesPullingTime="2.425997598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:51.797159484 +0000 UTC m=+289.363436721" watchObservedRunningTime="2026-06-12 18:42:51.797231727 +0000 UTC m=+289.363508965" Jun 12 18:42:54.705262 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.705224 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:42:54.710953 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.710933 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.713639 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.713619 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 12 18:42:54.714860 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714773 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 12 18:42:54.714860 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714790 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 12 18:42:54.714860 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714799 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b9sww\"" Jun 12 18:42:54.714860 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714816 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 12 18:42:54.714860 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714861 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 12 18:42:54.715116 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714868 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 12 18:42:54.715116 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.714820 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 12 18:42:54.718954 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.718930 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:42:54.812481 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.812481 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.812728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.812728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qmlv\" (UniqueName: \"kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.812728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.812728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.812608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.913834 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.913834 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914073 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914073 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qmlv\" (UniqueName: \"kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914073 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914073 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.913954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914625 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.914603 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914735 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.914715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.914774 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.914715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.916326 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.916291 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.916407 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.916391 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:54.928611 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:54.928592 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qmlv\" (UniqueName: \"kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv\") pod \"console-6dfb8dfc95-47pf7\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:55.019836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:55.019748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:42:55.154439 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:55.154257 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:42:55.158660 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:55.158623 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdf4c18_6294_45fd_aafd_3794eeac1f5e.slice/crio-97c54643ded7c32571a83ec90b26db720f51d975ecb3f844e1594fd2c7814b35 WatchSource:0}: Error finding container 97c54643ded7c32571a83ec90b26db720f51d975ecb3f844e1594fd2c7814b35: Status 404 returned error can't find the container with id 97c54643ded7c32571a83ec90b26db720f51d975ecb3f844e1594fd2c7814b35 Jun 12 18:42:55.746859 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:55.746811 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb8dfc95-47pf7" event={"ID":"dbdf4c18-6294-45fd-aafd-3794eeac1f5e","Type":"ContainerStarted","Data":"97c54643ded7c32571a83ec90b26db720f51d975ecb3f844e1594fd2c7814b35"} Jun 12 18:42:56.501076 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.501044 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:42:56.504445 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.504424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.512955 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.512920 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 12 18:42:56.514471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.514450 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:42:56.631893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.631854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.631893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.631894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btsxw\" (UniqueName: \"kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.632138 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.631972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.632138 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.631993 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.632138 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.632012 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.632138 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.632092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.632342 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.632156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733236 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733236 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btsxw\" (UniqueName: \"kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733496 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733272 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733496 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733496 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733496 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.733496 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.733383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.734096 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.734071 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.734192 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.734157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.734269 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.734248 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.734340 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.734301 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.735834 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.735810 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.735940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.735920 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.742057 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.742032 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btsxw\" (UniqueName: \"kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw\") pod \"console-7549f84959-lzz4p\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.814698 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.814614 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:42:56.961524 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:56.961494 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:42:56.966701 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:42:56.966673 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf76791_4b62_428c_bb0a_71f7aff8fde2.slice/crio-27961b1c76bb39182afca13b3a2d00114380ef4d1e4a8f2eaf1543665f35ef1b WatchSource:0}: Error finding container 27961b1c76bb39182afca13b3a2d00114380ef4d1e4a8f2eaf1543665f35ef1b: Status 404 returned error can't find the container with id 27961b1c76bb39182afca13b3a2d00114380ef4d1e4a8f2eaf1543665f35ef1b Jun 12 18:42:57.755360 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:57.755307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7549f84959-lzz4p" event={"ID":"6bf76791-4b62-428c-bb0a-71f7aff8fde2","Type":"ContainerStarted","Data":"27961b1c76bb39182afca13b3a2d00114380ef4d1e4a8f2eaf1543665f35ef1b"} Jun 12 18:42:58.761995 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:58.761957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb8dfc95-47pf7" event={"ID":"dbdf4c18-6294-45fd-aafd-3794eeac1f5e","Type":"ContainerStarted","Data":"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3"} Jun 12 18:42:58.764282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:58.763718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7549f84959-lzz4p" event={"ID":"6bf76791-4b62-428c-bb0a-71f7aff8fde2","Type":"ContainerStarted","Data":"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76"} Jun 12 18:42:58.804989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:58.804934 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfb8dfc95-47pf7" podStartSLOduration=1.294764626 podStartE2EDuration="4.804913571s" totalImagesPullingTime="3.510148945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:58.804368593 +0000 UTC m=+296.370645830" watchObservedRunningTime="2026-06-12 18:42:58.804913571 +0000 UTC m=+296.371190807" Jun 12 18:42:58.849780 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:58.849719 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7549f84959-lzz4p" podStartSLOduration=1.154785932 podStartE2EDuration="2.849699063s" totalImagesPullingTime="1.694913131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:42:58.848665857 +0000 UTC m=+296.414943104" watchObservedRunningTime="2026-06-12 18:42:58.849699063 +0000 UTC m=+296.415976304" Jun 12 18:42:58.895124 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:42:58.895082 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8k97v" podUID="dd0084c5-7e69-4534-8301-20b771c71427" Jun 12 18:42:59.375292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:42:59.375263 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:43:01.735961 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.735928 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zjnct" Jun 12 18:43:01.796612 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.796578 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99"] Jun 12 18:43:01.800965 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.800943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.805396 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.805371 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 12 18:43:01.808020 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.807974 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qh4md\"" Jun 12 18:43:01.810460 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.810440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 12 18:43:01.810589 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.810572 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 12 18:43:01.831773 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.831742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99"] Jun 12 18:43:01.852322 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.852290 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rm828"] Jun 12 18:43:01.855414 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.855395 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.859797 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.859773 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 12 18:43:01.859896 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.859863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 12 18:43:01.866544 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.866527 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6h9q4\"" Jun 12 18:43:01.869297 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.869281 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 12 18:43:01.879358 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.879305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.879469 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.879436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90dbec6c-b06a-4415-b931-9c568e40f8b2-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.879469 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.879458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.879597 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.879475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnthc\" (UniqueName: \"kubernetes.io/projected/90dbec6c-b06a-4415-b931-9c568e40f8b2-kube-api-access-jnthc\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.980670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.980670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90dbec6c-b06a-4415-b931-9c568e40f8b2-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnthc\" (UniqueName: \"kubernetes.io/projected/90dbec6c-b06a-4415-b931-9c568e40f8b2-kube-api-access-jnthc\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-wtmp\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-accelerators-collector-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-sys\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.980914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-tls\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.980972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-textfile\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.981004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-metrics-client-ca\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.981026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.981090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22nk\" (UniqueName: \"kubernetes.io/projected/42129e6f-63c9-49da-b4e9-1915e1bd8b30-kube-api-access-m22nk\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.981129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-root\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:01.981522 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.981505 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90dbec6c-b06a-4415-b931-9c568e40f8b2-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.983039 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.983021 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.983166 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.983147 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90dbec6c-b06a-4415-b931-9c568e40f8b2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:01.990533 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:01.990481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnthc\" (UniqueName: \"kubernetes.io/projected/90dbec6c-b06a-4415-b931-9c568e40f8b2-kube-api-access-jnthc\") pod \"openshift-state-metrics-6b8d965dd5-8qw99\" (UID: \"90dbec6c-b06a-4415-b931-9c568e40f8b2\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:02.082163 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-root\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082199 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-wtmp\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-accelerators-collector-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-sys\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-tls\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082258 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-root\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-textfile\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082348 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-sys\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-metrics-client-ca\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082435 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-wtmp\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m22nk\" (UniqueName: \"kubernetes.io/projected/42129e6f-63c9-49da-b4e9-1915e1bd8b30-kube-api-access-m22nk\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082878 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082732 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-textfile\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.082933 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.082890 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-accelerators-collector-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.083026 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.083007 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42129e6f-63c9-49da-b4e9-1915e1bd8b30-metrics-client-ca\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.085219 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.085193 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-tls\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.085343 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.085224 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42129e6f-63c9-49da-b4e9-1915e1bd8b30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.094509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.094481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22nk\" (UniqueName: \"kubernetes.io/projected/42129e6f-63c9-49da-b4e9-1915e1bd8b30-kube-api-access-m22nk\") pod \"node-exporter-rm828\" (UID: \"42129e6f-63c9-49da-b4e9-1915e1bd8b30\") " pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.110406 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.110384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" Jun 12 18:43:02.164476 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.164448 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rm828" Jun 12 18:43:02.178075 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:02.178027 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42129e6f_63c9_49da_b4e9_1915e1bd8b30.slice/crio-b404271ecabf5c442819e2969850fbdc64a7058a13736065a32d5234637ca0c4 WatchSource:0}: Error finding container b404271ecabf5c442819e2969850fbdc64a7058a13736065a32d5234637ca0c4: Status 404 returned error can't find the container with id b404271ecabf5c442819e2969850fbdc64a7058a13736065a32d5234637ca0c4 Jun 12 18:43:02.246545 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.246371 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99"] Jun 12 18:43:02.249138 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:02.249107 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90dbec6c_b06a_4415_b931_9c568e40f8b2.slice/crio-3995e7f18c29b94919942388c6b06dcbc122ec59135e4a9f9a8e42667a5efb99 WatchSource:0}: Error finding container 3995e7f18c29b94919942388c6b06dcbc122ec59135e4a9f9a8e42667a5efb99: Status 404 returned error can't find the container with id 3995e7f18c29b94919942388c6b06dcbc122ec59135e4a9f9a8e42667a5efb99 Jun 12 18:43:02.781086 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.781049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rm828" event={"ID":"42129e6f-63c9-49da-b4e9-1915e1bd8b30","Type":"ContainerStarted","Data":"b404271ecabf5c442819e2969850fbdc64a7058a13736065a32d5234637ca0c4"} Jun 12 18:43:02.783422 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.783389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" event={"ID":"90dbec6c-b06a-4415-b931-9c568e40f8b2","Type":"ContainerStarted","Data":"a8f4f907acb3fe38138ae67da0ff4ede4ecd14222cac80e07fc233881e264b6d"} Jun 12 18:43:02.783572 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.783459 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" event={"ID":"90dbec6c-b06a-4415-b931-9c568e40f8b2","Type":"ContainerStarted","Data":"6bc8df720bee8a1671d12b7deff94a8b93579b8b446ad15dc5ebc0b432f512e2"} Jun 12 18:43:02.783572 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.783493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" event={"ID":"90dbec6c-b06a-4415-b931-9c568e40f8b2","Type":"ContainerStarted","Data":"3995e7f18c29b94919942388c6b06dcbc122ec59135e4a9f9a8e42667a5efb99"} Jun 12 18:43:02.957569 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.957388 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.962140 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.964923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.964945 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.964966 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.964929 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965096 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965175 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965497 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965551 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965565 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.965834 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wj7mv\"" Jun 12 18:43:02.983288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:02.978633 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:43:03.091925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.091857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.091925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.091894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.091925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.091918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092220 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.092450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.092351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt66n\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193573 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193540 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.193983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.193954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt66n\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.194992 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.194847 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.195983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.195219 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.195983 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.195703 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.197277 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.197253 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.197637 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.197580 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.197947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.197926 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.197947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.197946 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.199077 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.199017 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.199504 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.199423 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.199631 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.199533 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.199881 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.199856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.199961 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.199917 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.206406 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.206387 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt66n\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n\") pod \"alertmanager-main-0\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.274461 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.274427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:43:03.397935 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.397908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:43:03.398483 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.398451 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:43:03.419370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.419347 2577 kubelet.go:1648] "Image garbage collection succeeded" Jun 12 18:43:03.547153 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.546956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:43:03.549422 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:03.549393 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc633f0a7_cd20_4bf7_8903_d727f15bbdf8.slice/crio-96ed9603d34e3672ea1d7bc5905541044b5fafee70326f7e0eaf4ef5ae9b1284 WatchSource:0}: Error finding container 96ed9603d34e3672ea1d7bc5905541044b5fafee70326f7e0eaf4ef5ae9b1284: Status 404 returned error can't find the container with id 96ed9603d34e3672ea1d7bc5905541044b5fafee70326f7e0eaf4ef5ae9b1284 Jun 12 18:43:03.788882 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.788845 2577 generic.go:356] "Generic (PLEG): container finished" podID="42129e6f-63c9-49da-b4e9-1915e1bd8b30" containerID="bb2a796bc40c902f9dcbf32fd0e121301022ae791a192a063df70100d95d4bc9" exitCode=0 Jun 12 18:43:03.789363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.788923 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rm828" event={"ID":"42129e6f-63c9-49da-b4e9-1915e1bd8b30","Type":"ContainerDied","Data":"bb2a796bc40c902f9dcbf32fd0e121301022ae791a192a063df70100d95d4bc9"} Jun 12 18:43:03.791192 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.791165 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" event={"ID":"90dbec6c-b06a-4415-b931-9c568e40f8b2","Type":"ContainerStarted","Data":"2023f0aba5ab73bfbc3e6731b081b5fa2d4f4b758f2b5c8906cb8dc28a3ef1b1"} Jun 12 18:43:03.792397 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.792374 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"96ed9603d34e3672ea1d7bc5905541044b5fafee70326f7e0eaf4ef5ae9b1284"} Jun 12 18:43:03.831883 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:03.831832 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-8qw99" podStartSLOduration=1.8096001720000001 podStartE2EDuration="2.831810404s" totalImagesPullingTime="1.022210232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:03.830042059 +0000 UTC m=+301.396319295" watchObservedRunningTime="2026-06-12 18:43:03.831810404 +0000 UTC m=+301.398087639" Jun 12 18:43:04.798247 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.798214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rm828" event={"ID":"42129e6f-63c9-49da-b4e9-1915e1bd8b30","Type":"ContainerStarted","Data":"452906b0ce033f2d52fb15de2caf400e7197a8a275a5d9c6f38fd4ee47026274"} Jun 12 18:43:04.798706 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.798296 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rm828" event={"ID":"42129e6f-63c9-49da-b4e9-1915e1bd8b30","Type":"ContainerStarted","Data":"5412ad9a8fc2f4864a15471a89ada8b2aba1b06d3f4db104b6f74d002803c69a"} Jun 12 18:43:04.799773 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.799704 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5" exitCode=0 Jun 12 18:43:04.799869 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.799785 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5"} Jun 12 18:43:04.800893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.800876 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 18:43:04.821397 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:04.821343 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rm828" podStartSLOduration=2.541634184 podStartE2EDuration="3.821324514s" totalImagesPullingTime="1.200439265s" totalInitContainerRuntime="79.251065ms" isStatefulPod=true podCreationTimestamp="2026-06-12 18:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:04.819950333 +0000 UTC m=+302.386227578" watchObservedRunningTime="2026-06-12 18:43:04.821324514 +0000 UTC m=+302.387601741" Jun 12 18:43:05.020371 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:05.020341 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:05.020560 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:05.020405 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:05.025275 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:05.025245 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:05.807094 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:05.807062 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:06.295908 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.295869 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-dbb6b689d-dcs28"] Jun 12 18:43:06.300339 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.300298 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.303097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.303076 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5q68h\"" Jun 12 18:43:06.303232 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.303109 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 12 18:43:06.303406 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.303377 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 12 18:43:06.304350 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.304295 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 12 18:43:06.304350 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.304332 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3k13jl472iknn\"" Jun 12 18:43:06.304523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.304372 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 12 18:43:06.311525 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.311505 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-dbb6b689d-dcs28"] Jun 12 18:43:06.429008 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.428969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbdgp\" (UniqueName: \"kubernetes.io/projected/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-kube-api-access-cbdgp\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429192 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-tls\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429192 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-client-certs\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429334 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-metrics-server-audit-profiles\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429334 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-client-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429334 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429276 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.429479 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.429365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-audit-log\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.524679 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.524647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv"] Jun 12 18:43:06.527749 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.527723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.530386 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530361 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 12 18:43:06.530513 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-client-certs\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530513 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dsvc4\"" Jun 12 18:43:06.530513 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-metrics-server-audit-profiles\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530693 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-client-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530693 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530693 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530671 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-audit-log\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530842 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbdgp\" (UniqueName: \"kubernetes.io/projected/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-kube-api-access-cbdgp\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.530842 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.530766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-tls\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.531742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.531330 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-audit-log\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.531742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.531644 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-metrics-server-audit-profiles\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.531742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.531642 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.533092 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.533070 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-client-certs\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.533356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.533339 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-client-ca-bundle\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.533484 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.533463 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-secret-metrics-server-tls\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.540699 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.540677 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv"] Jun 12 18:43:06.544011 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.543988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbdgp\" (UniqueName: \"kubernetes.io/projected/ddc1a0b5-3c29-4316-91e9-9fcb1ed46980-kube-api-access-cbdgp\") pod \"metrics-server-dbb6b689d-dcs28\" (UID: \"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980\") " pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.613005 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.612985 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:06.631490 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.631468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9hx\" (UniqueName: \"kubernetes.io/projected/5e187eea-f7e6-4123-941c-c776e0857db7-kube-api-access-gv9hx\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.631611 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.631568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.732471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.732342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9hx\" (UniqueName: \"kubernetes.io/projected/5e187eea-f7e6-4123-941c-c776e0857db7-kube-api-access-gv9hx\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.732471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.732471 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.732704 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:06.732632 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 12 18:43:06.732704 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:06.732699 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert podName:5e187eea-f7e6-4123-941c-c776e0857db7 nodeName:}" failed. No retries permitted until 2026-06-12 18:43:07.232677891 +0000 UTC m=+304.798955107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert") pod "monitoring-plugin-589576dfc7-t6qhv" (UID: "5e187eea-f7e6-4123-941c-c776e0857db7") : secret "monitoring-plugin-cert" not found Jun 12 18:43:06.741610 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.741557 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9hx\" (UniqueName: \"kubernetes.io/projected/5e187eea-f7e6-4123-941c-c776e0857db7-kube-api-access-gv9hx\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:06.767572 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.767544 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-dbb6b689d-dcs28"] Jun 12 18:43:06.771324 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:06.771289 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc1a0b5_3c29_4316_91e9_9fcb1ed46980.slice/crio-a9e93b5fa29e6227864b5f2434db556791f8bcd04f902cb584e356cb42e04d19 WatchSource:0}: Error finding container a9e93b5fa29e6227864b5f2434db556791f8bcd04f902cb584e356cb42e04d19: Status 404 returned error can't find the container with id a9e93b5fa29e6227864b5f2434db556791f8bcd04f902cb584e356cb42e04d19 Jun 12 18:43:06.810152 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.810117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54"} Jun 12 18:43:06.810517 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.810267 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8"} Jun 12 18:43:06.813650 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.813624 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" event={"ID":"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980","Type":"ContainerStarted","Data":"a9e93b5fa29e6227864b5f2434db556791f8bcd04f902cb584e356cb42e04d19"} Jun 12 18:43:06.815452 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.815410 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:06.816734 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.816185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:06.821014 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:06.820992 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:07.236379 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.236345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:07.238659 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.238624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e187eea-f7e6-4123-941c-c776e0857db7-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-t6qhv\" (UID: \"5e187eea-f7e6-4123-941c-c776e0857db7\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:07.458122 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.458085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:07.616117 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.616081 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv"] Jun 12 18:43:07.619345 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:07.619302 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e187eea_f7e6_4123_941c_c776e0857db7.slice/crio-983cc3e2ef248778ba7603b7f5b8920c819235e25a49981c322f4d8d8d755342 WatchSource:0}: Error finding container 983cc3e2ef248778ba7603b7f5b8920c819235e25a49981c322f4d8d8d755342: Status 404 returned error can't find the container with id 983cc3e2ef248778ba7603b7f5b8920c819235e25a49981c322f4d8d8d755342 Jun 12 18:43:07.717197 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.717166 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:43:07.756141 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.756106 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:07.759666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.759638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.771040 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.771009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:07.821112 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.821068 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830"} Jun 12 18:43:07.821562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.821155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7"} Jun 12 18:43:07.821562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.821201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926"} Jun 12 18:43:07.822721 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.822665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" event={"ID":"5e187eea-f7e6-4123-941c-c776e0857db7","Type":"ContainerStarted","Data":"983cc3e2ef248778ba7603b7f5b8920c819235e25a49981c322f4d8d8d755342"} Jun 12 18:43:07.827191 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.827168 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:07.842877 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.842833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.842877 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.842866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkrsk\" (UniqueName: \"kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.843082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.842959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.843082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.842992 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.843082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.843020 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.843270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.843094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.843270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.843181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944055 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkrsk\" (UniqueName: \"kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944437 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944437 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944437 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.944894 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.944838 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.945147 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.945103 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.945456 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.945427 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.945623 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.945607 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.947046 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.947028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.947637 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.947616 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:07.960974 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:07.960951 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkrsk\" (UniqueName: \"kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk\") pod \"console-5cf968585b-hwlnb\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:08.072028 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.071948 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:08.469957 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.469925 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:08.473051 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:08.473023 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834205ae_ff46_435d_83ae_63db35ea33ff.slice/crio-121b8b66fcba8cc37e0b3854ae4f34c0b3c661ca597111697ea8e1faba2b3e79 WatchSource:0}: Error finding container 121b8b66fcba8cc37e0b3854ae4f34c0b3c661ca597111697ea8e1faba2b3e79: Status 404 returned error can't find the container with id 121b8b66fcba8cc37e0b3854ae4f34c0b3c661ca597111697ea8e1faba2b3e79 Jun 12 18:43:08.832430 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.832389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerStarted","Data":"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e"} Jun 12 18:43:08.834454 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.834424 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" event={"ID":"ddc1a0b5-3c29-4316-91e9-9fcb1ed46980","Type":"ContainerStarted","Data":"bdb5db2cba7b050be6b980f1dfe2c7fc60bc3307bd975796da9fafaa80261872"} Jun 12 18:43:08.836374 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.836342 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf968585b-hwlnb" event={"ID":"834205ae-ff46-435d-83ae-63db35ea33ff","Type":"ContainerStarted","Data":"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d"} Jun 12 18:43:08.836374 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.836392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf968585b-hwlnb" event={"ID":"834205ae-ff46-435d-83ae-63db35ea33ff","Type":"ContainerStarted","Data":"121b8b66fcba8cc37e0b3854ae4f34c0b3c661ca597111697ea8e1faba2b3e79"} Jun 12 18:43:08.882389 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.882329 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.69227995 podStartE2EDuration="6.882292949s" totalImagesPullingTime="4.173057334s" totalInitContainerRuntime="16.955665ms" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:08.880899017 +0000 UTC m=+306.447176253" watchObservedRunningTime="2026-06-12 18:43:08.882292949 +0000 UTC m=+306.448570187" Jun 12 18:43:08.998666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:08.998604 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" podStartSLOduration=1.453077828 podStartE2EDuration="2.998580202s" totalImagesPullingTime="1.545502374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:08.965426641 +0000 UTC m=+306.531703877" watchObservedRunningTime="2026-06-12 18:43:08.998580202 +0000 UTC m=+306.564857442" Jun 12 18:43:09.000096 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:09.000050 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cf968585b-hwlnb" podStartSLOduration=2.000036144 podStartE2EDuration="2.000036144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:08.996164493 +0000 UTC m=+306.562441743" watchObservedRunningTime="2026-06-12 18:43:09.000036144 +0000 UTC m=+306.566313408" Jun 12 18:43:09.841950 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:09.841916 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" event={"ID":"5e187eea-f7e6-4123-941c-c776e0857db7","Type":"ContainerStarted","Data":"0b90bc715321682d276c6bedab75c9b304dddb9d53236edda275adffe7132798"} Jun 12 18:43:09.974106 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:09.974055 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" podStartSLOduration=2.318173011 podStartE2EDuration="3.974039553s" totalImagesPullingTime="1.655866542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:09.968035883 +0000 UTC m=+307.534313119" watchObservedRunningTime="2026-06-12 18:43:09.974039553 +0000 UTC m=+307.540316788" Jun 12 18:43:10.846124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:10.846090 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:10.851515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:10.851490 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-t6qhv" Jun 12 18:43:13.893709 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:13.893678 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:43:14.389768 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.389732 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" containerName="registry" containerID="cri-o://22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97" gracePeriod=30 Jun 12 18:43:14.498433 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.498402 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:14.526189 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.526162 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:43:14.529584 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.529566 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.582907 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.582867 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:43:14.654677 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.654653 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:43:14.711921 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.711886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.711934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.711973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.712037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.712060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9cp8\" (UniqueName: \"kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.712093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.712296 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.712146 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813044 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813078 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813145 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813163 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813188 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813205 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvm2p\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813543 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813343 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813543 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") pod \"f2eaf655-5707-4380-996e-15531eecd4a3\" (UID: \"f2eaf655-5707-4380-996e-15531eecd4a3\") " Jun 12 18:43:14.813644 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813644 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813743 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813586 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:14.813743 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9cp8\" (UniqueName: \"kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813798 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:14.813977 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.813977 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.813890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.814075 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.814047 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-trusted-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.814075 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.814066 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2eaf655-5707-4380-996e-15531eecd4a3-registry-certificates\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.814623 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.814598 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.814730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.814710 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.815422 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.815396 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.816266 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.816241 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.817017 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.816990 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.817389 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817366 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:14.817490 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817407 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:14.817554 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817529 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:14.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817653 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:14.817748 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817725 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p" (OuterVolumeSpecName: "kube-api-access-rvm2p") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "kube-api-access-rvm2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:14.817938 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.817921 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.823589 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.823566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9cp8\" (UniqueName: \"kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8\") pod \"console-cd59b76b7-t2zfr\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.823775 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.823754 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f2eaf655-5707-4380-996e-15531eecd4a3" (UID: "f2eaf655-5707-4380-996e-15531eecd4a3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:43:14.839283 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.839262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:14.865169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.865134 2577 generic.go:356] "Generic (PLEG): container finished" podID="f2eaf655-5707-4380-996e-15531eecd4a3" containerID="22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97" exitCode=0 Jun 12 18:43:14.865356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.865197 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" Jun 12 18:43:14.865356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.865233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" event={"ID":"f2eaf655-5707-4380-996e-15531eecd4a3","Type":"ContainerDied","Data":"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97"} Jun 12 18:43:14.865356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.865302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676759d85f-5xw5b" event={"ID":"f2eaf655-5707-4380-996e-15531eecd4a3","Type":"ContainerDied","Data":"ca13e832b4c86e4083b2acd132cea4986c528497271e585a674228fe5c4e864b"} Jun 12 18:43:14.865555 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.865367 2577 scope.go:117] "RemoveContainer" containerID="22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97" Jun 12 18:43:14.875962 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.875930 2577 scope.go:117] "RemoveContainer" containerID="22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97" Jun 12 18:43:14.876267 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:14.876247 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97\": container with ID starting with 22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97 not found: ID does not exist" containerID="22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97" Jun 12 18:43:14.876365 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.876275 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97"} err="failed to get container status \"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97\": rpc error: code = NotFound desc = could not find container \"22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97\": container with ID starting with 22b7e26104c51aa0cc7b3d4002223ae975a3aaf279313314070dfe9249a1ae97 not found: ID does not exist" Jun 12 18:43:14.889757 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.889695 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:43:14.900925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.900823 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-676759d85f-5xw5b"] Jun 12 18:43:14.915300 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915268 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2eaf655-5707-4380-996e-15531eecd4a3-ca-trust-extracted\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.915467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915303 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-image-registry-private-configuration\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.915467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915355 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-bound-sa-token\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.915467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915368 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvm2p\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-kube-api-access-rvm2p\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.915467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915381 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2eaf655-5707-4380-996e-15531eecd4a3-registry-tls\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:14.915467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:14.915395 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2eaf655-5707-4380-996e-15531eecd4a3-installation-pull-secrets\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:15.189518 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:15.189447 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:43:15.192127 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:43:15.192101 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192bbea2_dcfc_4df5_baea_366c34eb9043.slice/crio-0b37d97f33785a3de096efb97a9a6a63407151446344a8ddb81fbd2d4352f716 WatchSource:0}: Error finding container 0b37d97f33785a3de096efb97a9a6a63407151446344a8ddb81fbd2d4352f716: Status 404 returned error can't find the container with id 0b37d97f33785a3de096efb97a9a6a63407151446344a8ddb81fbd2d4352f716 Jun 12 18:43:15.871736 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:15.871703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd59b76b7-t2zfr" event={"ID":"192bbea2-dcfc-4df5-baea-366c34eb9043","Type":"ContainerStarted","Data":"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664"} Jun 12 18:43:15.871890 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:15.871751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd59b76b7-t2zfr" event={"ID":"192bbea2-dcfc-4df5-baea-366c34eb9043","Type":"ContainerStarted","Data":"0b37d97f33785a3de096efb97a9a6a63407151446344a8ddb81fbd2d4352f716"} Jun 12 18:43:15.894017 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:15.893965 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd59b76b7-t2zfr" podStartSLOduration=1.8939505699999999 podStartE2EDuration="1.89395057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:43:15.893478728 +0000 UTC m=+313.459755985" watchObservedRunningTime="2026-06-12 18:43:15.89395057 +0000 UTC m=+313.460227804" Jun 12 18:43:16.903269 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:16.903232 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" path="/var/lib/kubelet/pods/f2eaf655-5707-4380-996e-15531eecd4a3/volumes" Jun 12 18:43:18.072800 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:18.072764 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:24.840029 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:24.839995 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:24.840447 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:24.840053 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:24.844879 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:24.844857 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:24.912542 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:24.912511 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:43:24.964914 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:24.964879 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:43:26.613455 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:26.613428 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:26.613842 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:26.613478 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:33.860777 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:33.860710 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7549f84959-lzz4p" podUID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" containerName="console" containerID="cri-o://7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76" gracePeriod=15 Jun 12 18:43:34.141960 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.141929 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:34.177789 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177759 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.177946 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177798 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.177946 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177834 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.177946 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177856 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.177946 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177878 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.177946 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177904 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.178206 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.177972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btsxw\" (UniqueName: \"kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw\") pod \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\" (UID: \"6bf76791-4b62-428c-bb0a-71f7aff8fde2\") " Jun 12 18:43:34.178349 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.178279 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config" (OuterVolumeSpecName: "console-config") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:34.178349 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.178288 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:34.178624 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.178390 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca" (OuterVolumeSpecName: "service-ca") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:34.178624 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.178571 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:34.180648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.180618 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:34.180759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.180654 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:34.180759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.180711 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw" (OuterVolumeSpecName: "kube-api-access-btsxw") pod "6bf76791-4b62-428c-bb0a-71f7aff8fde2" (UID: "6bf76791-4b62-428c-bb0a-71f7aff8fde2"). InnerVolumeSpecName "kube-api-access-btsxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:34.279234 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279199 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btsxw\" (UniqueName: \"kubernetes.io/projected/6bf76791-4b62-428c-bb0a-71f7aff8fde2-kube-api-access-btsxw\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279234 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279227 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-oauth-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279234 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279236 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-trusted-ca-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279245 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279254 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bf76791-4b62-428c-bb0a-71f7aff8fde2-console-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279263 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-oauth-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.279480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.279271 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bf76791-4b62-428c-bb0a-71f7aff8fde2-service-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:34.946744 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.946710 2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" containerID="7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76" exitCode=0 Jun 12 18:43:34.947130 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.946769 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7549f84959-lzz4p" Jun 12 18:43:34.947130 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.946800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7549f84959-lzz4p" event={"ID":"6bf76791-4b62-428c-bb0a-71f7aff8fde2","Type":"ContainerDied","Data":"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76"} Jun 12 18:43:34.947130 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.946850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7549f84959-lzz4p" event={"ID":"6bf76791-4b62-428c-bb0a-71f7aff8fde2","Type":"ContainerDied","Data":"27961b1c76bb39182afca13b3a2d00114380ef4d1e4a8f2eaf1543665f35ef1b"} Jun 12 18:43:34.947130 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.946870 2577 scope.go:117] "RemoveContainer" containerID="7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76" Jun 12 18:43:34.954751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.954735 2577 scope.go:117] "RemoveContainer" containerID="7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76" Jun 12 18:43:34.955017 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:34.955001 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76\": container with ID starting with 7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76 not found: ID does not exist" containerID="7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76" Jun 12 18:43:34.955066 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.955024 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76"} err="failed to get container status \"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76\": rpc error: code = NotFound desc = could not find container \"7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76\": container with ID starting with 7a4dcf3f5336ed2e1a54dafc9a5ccbd2daba580b6b8978a87cd8c271df146c76 not found: ID does not exist" Jun 12 18:43:34.964095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.964075 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:43:34.969645 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:34.969625 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7549f84959-lzz4p"] Jun 12 18:43:36.899233 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:36.899204 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" path="/var/lib/kubelet/pods/6bf76791-4b62-428c-bb0a-71f7aff8fde2/volumes" Jun 12 18:43:39.518657 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.518614 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cf968585b-hwlnb" podUID="834205ae-ff46-435d-83ae-63db35ea33ff" containerName="console" containerID="cri-o://6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d" gracePeriod=15 Jun 12 18:43:39.801841 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.801814 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:39.931512 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931477 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931528 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931545 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931568 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931616 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkrsk\" (UniqueName: \"kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931632 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.931700 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931656 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config\") pod \"834205ae-ff46-435d-83ae-63db35ea33ff\" (UID: \"834205ae-ff46-435d-83ae-63db35ea33ff\") " Jun 12 18:43:39.932014 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931955 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:39.932014 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.931976 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:39.932139 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.932120 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config" (OuterVolumeSpecName: "console-config") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:39.932216 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.932184 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:39.933935 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.933911 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk" (OuterVolumeSpecName: "kube-api-access-xkrsk") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "kube-api-access-xkrsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:39.934032 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.933958 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:39.934032 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.933993 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "834205ae-ff46-435d-83ae-63db35ea33ff" (UID: "834205ae-ff46-435d-83ae-63db35ea33ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:39.969131 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.969100 2577 generic.go:356] "Generic (PLEG): container finished" podID="834205ae-ff46-435d-83ae-63db35ea33ff" containerID="6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d" exitCode=0 Jun 12 18:43:39.969306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.969141 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf968585b-hwlnb" Jun 12 18:43:39.969306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.969180 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf968585b-hwlnb" event={"ID":"834205ae-ff46-435d-83ae-63db35ea33ff","Type":"ContainerDied","Data":"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d"} Jun 12 18:43:39.969306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.969222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf968585b-hwlnb" event={"ID":"834205ae-ff46-435d-83ae-63db35ea33ff","Type":"ContainerDied","Data":"121b8b66fcba8cc37e0b3854ae4f34c0b3c661ca597111697ea8e1faba2b3e79"} Jun 12 18:43:39.969306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.969247 2577 scope.go:117] "RemoveContainer" containerID="6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d" Jun 12 18:43:39.977778 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.977758 2577 scope.go:117] "RemoveContainer" containerID="6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d" Jun 12 18:43:39.978058 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:39.978034 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d\": container with ID starting with 6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d not found: ID does not exist" containerID="6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d" Jun 12 18:43:39.978148 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.978064 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d"} err="failed to get container status \"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d\": rpc error: code = NotFound desc = could not find container \"6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d\": container with ID starting with 6440db940653b822068768072e7d54b5072d3a6b0058385ec7ae1c894383f43d not found: ID does not exist" Jun 12 18:43:39.990035 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.990012 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:39.994319 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:39.994288 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cf968585b-hwlnb"] Jun 12 18:43:40.032466 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032435 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-trusted-ca-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032466 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032463 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032476 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-service-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032486 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-console-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032494 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkrsk\" (UniqueName: \"kubernetes.io/projected/834205ae-ff46-435d-83ae-63db35ea33ff-kube-api-access-xkrsk\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032502 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834205ae-ff46-435d-83ae-63db35ea33ff-oauth-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.032666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.032511 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834205ae-ff46-435d-83ae-63db35ea33ff-console-oauth-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:40.898508 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:40.898473 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834205ae-ff46-435d-83ae-63db35ea33ff" path="/var/lib/kubelet/pods/834205ae-ff46-435d-83ae-63db35ea33ff/volumes" Jun 12 18:43:46.619591 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:46.619561 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:46.623405 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:46.623381 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-dbb6b689d-dcs28" Jun 12 18:43:49.986665 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:49.986624 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6dfb8dfc95-47pf7" podUID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" containerName="console" containerID="cri-o://aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3" gracePeriod=15 Jun 12 18:43:50.281201 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.281179 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:50.419015 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.418981 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419196 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419027 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419196 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419076 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419286 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419258 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419406 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419386 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qmlv\" (UniqueName: \"kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419482 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419436 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert\") pod \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\" (UID: \"dbdf4c18-6294-45fd-aafd-3794eeac1f5e\") " Jun 12 18:43:50.419539 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419485 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config" (OuterVolumeSpecName: "console-config") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:50.419539 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419490 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca" (OuterVolumeSpecName: "service-ca") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:50.419688 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419671 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-service-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:50.419732 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419692 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:50.419825 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.419802 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:43:50.421303 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.421282 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:50.421498 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.421478 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv" (OuterVolumeSpecName: "kube-api-access-5qmlv") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "kube-api-access-5qmlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:43:50.421542 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.421509 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dbdf4c18-6294-45fd-aafd-3794eeac1f5e" (UID: "dbdf4c18-6294-45fd-aafd-3794eeac1f5e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:43:50.521080 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.520987 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:50.521080 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.521012 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-console-oauth-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:50.521080 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.521022 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qmlv\" (UniqueName: \"kubernetes.io/projected/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-kube-api-access-5qmlv\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:50.521080 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:50.521032 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbdf4c18-6294-45fd-aafd-3794eeac1f5e-oauth-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:43:51.014216 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.014183 2577 generic.go:356] "Generic (PLEG): container finished" podID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" containerID="aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3" exitCode=0 Jun 12 18:43:51.014660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.014232 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb8dfc95-47pf7" Jun 12 18:43:51.014660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.014265 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb8dfc95-47pf7" event={"ID":"dbdf4c18-6294-45fd-aafd-3794eeac1f5e","Type":"ContainerDied","Data":"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3"} Jun 12 18:43:51.014660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.014336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb8dfc95-47pf7" event={"ID":"dbdf4c18-6294-45fd-aafd-3794eeac1f5e","Type":"ContainerDied","Data":"97c54643ded7c32571a83ec90b26db720f51d975ecb3f844e1594fd2c7814b35"} Jun 12 18:43:51.014660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.014359 2577 scope.go:117] "RemoveContainer" containerID="aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3" Jun 12 18:43:51.022179 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.022163 2577 scope.go:117] "RemoveContainer" containerID="aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3" Jun 12 18:43:51.022449 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:43:51.022431 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3\": container with ID starting with aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3 not found: ID does not exist" containerID="aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3" Jun 12 18:43:51.022531 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.022456 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3"} err="failed to get container status \"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3\": rpc error: code = NotFound desc = could not find container \"aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3\": container with ID starting with aa64127c2058e2d7289cc171fa312a79c6d9fb51ee0231ebd6c39b0cd3f841c3 not found: ID does not exist" Jun 12 18:43:51.034119 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.034095 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:43:51.038174 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:51.038158 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfb8dfc95-47pf7"] Jun 12 18:43:52.898821 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:43:52.898710 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" path="/var/lib/kubelet/pods/dbdf4c18-6294-45fd-aafd-3794eeac1f5e/volumes" Jun 12 18:44:16.847603 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:16.847566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:44:16.849882 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:16.849859 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd0084c5-7e69-4534-8301-20b771c71427-metrics-certs\") pod \"network-metrics-daemon-8k97v\" (UID: \"dd0084c5-7e69-4534-8301-20b771c71427\") " pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:44:16.898665 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:16.898644 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lhhs9\"" Jun 12 18:44:16.905050 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:16.905036 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k97v" Jun 12 18:44:17.245261 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:17.245237 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8k97v"] Jun 12 18:44:17.247536 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:44:17.247502 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0084c5_7e69_4534_8301_20b771c71427.slice/crio-0e70ffa7916a28cff3e04e7b11762418b2d559f9fbedbe98046a7c63751ec4ab WatchSource:0}: Error finding container 0e70ffa7916a28cff3e04e7b11762418b2d559f9fbedbe98046a7c63751ec4ab: Status 404 returned error can't find the container with id 0e70ffa7916a28cff3e04e7b11762418b2d559f9fbedbe98046a7c63751ec4ab Jun 12 18:44:18.119526 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:18.119492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k97v" event={"ID":"dd0084c5-7e69-4534-8301-20b771c71427","Type":"ContainerStarted","Data":"0e70ffa7916a28cff3e04e7b11762418b2d559f9fbedbe98046a7c63751ec4ab"} Jun 12 18:44:19.124616 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:19.124583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k97v" event={"ID":"dd0084c5-7e69-4534-8301-20b771c71427","Type":"ContainerStarted","Data":"6b838d5485858369c6d34afd2ffbb39eccb08b3c87b1f1c7bfd97120b2cd3e3b"} Jun 12 18:44:19.124988 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:19.124635 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k97v" event={"ID":"dd0084c5-7e69-4534-8301-20b771c71427","Type":"ContainerStarted","Data":"353dfcd0afac357df67ffc5143ede8689fca87e3945d563ce2018d6974cf7e9c"} Jun 12 18:44:19.141052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:19.141004 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8k97v" podStartSLOduration=375.235549994 podStartE2EDuration="6m16.140990551s" totalImagesPullingTime="905.440557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:44:19.14090849 +0000 UTC m=+376.707185728" watchObservedRunningTime="2026-06-12 18:44:19.140990551 +0000 UTC m=+376.707267785" Jun 12 18:44:22.363729 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.363687 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364092 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="alertmanager" containerID="cri-o://f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" gracePeriod=120 Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364148 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-metric" containerID="cri-o://570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" gracePeriod=120 Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364204 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="prom-label-proxy" containerID="cri-o://7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" gracePeriod=120 Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364214 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy" containerID="cri-o://6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" gracePeriod=120 Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364233 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="config-reloader" containerID="cri-o://60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" gracePeriod=120 Jun 12 18:44:22.364376 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:22.364204 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-web" containerID="cri-o://6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" gracePeriod=120 Jun 12 18:44:23.144088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144053 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" exitCode=0 Jun 12 18:44:23.144088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144078 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" exitCode=0 Jun 12 18:44:23.144088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144088 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" exitCode=0 Jun 12 18:44:23.144088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144095 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" exitCode=0 Jun 12 18:44:23.144390 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144129 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e"} Jun 12 18:44:23.144390 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926"} Jun 12 18:44:23.144390 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54"} Jun 12 18:44:23.144390 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.144251 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8"} Jun 12 18:44:23.703868 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.703845 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:23.816233 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816200 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816233 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816233 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816261 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816287 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816358 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816387 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816446 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816471 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt66n\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816499 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816530 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816561 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816600 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816637 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy\") pod \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\" (UID: \"c633f0a7-cd20-4bf7-8903-d727f15bbdf8\") " Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816662 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816676 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:44:23.816794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816681 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:44:23.817190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816880 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.817190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816899 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-metrics-client-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.817190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.816914 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-alertmanager-main-db\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.819766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.819644 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out" (OuterVolumeSpecName: "config-out") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:44:23.819766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.819718 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.819766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.819737 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:44:23.819766 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.819742 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n" (OuterVolumeSpecName: "kube-api-access-mt66n") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "kube-api-access-mt66n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:44:23.820017 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.819982 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume" (OuterVolumeSpecName: "config-volume") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.820371 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.820340 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.820616 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.820590 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.820944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.820925 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.824580 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.824559 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.830684 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.830664 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config" (OuterVolumeSpecName: "web-config") pod "c633f0a7-cd20-4bf7-8903-d727f15bbdf8" (UID: "c633f0a7-cd20-4bf7-8903-d727f15bbdf8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:44:23.917375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917326 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917371 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-tls-assets\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917401 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-web-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917410 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917420 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-cluster-tls-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917429 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt66n\" (UniqueName: \"kubernetes.io/projected/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-kube-api-access-mt66n\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917438 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917448 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-out\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917456 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-secret-alertmanager-main-tls\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:23.917565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:23.917464 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c633f0a7-cd20-4bf7-8903-d727f15bbdf8-config-volume\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:44:24.152877 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.152845 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" exitCode=0 Jun 12 18:44:24.152877 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.152870 2577 generic.go:356] "Generic (PLEG): container finished" podID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerID="6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" exitCode=0 Jun 12 18:44:24.153132 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.152916 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7"} Jun 12 18:44:24.153132 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.152947 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.153132 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.152987 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830"} Jun 12 18:44:24.153132 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.153014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c633f0a7-cd20-4bf7-8903-d727f15bbdf8","Type":"ContainerDied","Data":"96ed9603d34e3672ea1d7bc5905541044b5fafee70326f7e0eaf4ef5ae9b1284"} Jun 12 18:44:24.153132 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.153044 2577 scope.go:117] "RemoveContainer" containerID="7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" Jun 12 18:44:24.160616 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.160496 2577 scope.go:117] "RemoveContainer" containerID="570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" Jun 12 18:44:24.167765 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.167750 2577 scope.go:117] "RemoveContainer" containerID="6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" Jun 12 18:44:24.174527 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.174512 2577 scope.go:117] "RemoveContainer" containerID="6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" Jun 12 18:44:24.180898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.180880 2577 scope.go:117] "RemoveContainer" containerID="60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" Jun 12 18:44:24.187282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.187251 2577 scope.go:117] "RemoveContainer" containerID="f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" Jun 12 18:44:24.194112 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.194093 2577 scope.go:117] "RemoveContainer" containerID="ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5" Jun 12 18:44:24.196179 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.196149 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:24.201193 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201178 2577 scope.go:117] "RemoveContainer" containerID="7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" Jun 12 18:44:24.201462 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.201437 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e\": container with ID starting with 7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e not found: ID does not exist" containerID="7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" Jun 12 18:44:24.201571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201468 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e"} err="failed to get container status \"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e\": rpc error: code = NotFound desc = could not find container \"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e\": container with ID starting with 7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e not found: ID does not exist" Jun 12 18:44:24.201571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201486 2577 scope.go:117] "RemoveContainer" containerID="570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" Jun 12 18:44:24.201715 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.201698 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7\": container with ID starting with 570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7 not found: ID does not exist" containerID="570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" Jun 12 18:44:24.201751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201717 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7"} err="failed to get container status \"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7\": rpc error: code = NotFound desc = could not find container \"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7\": container with ID starting with 570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7 not found: ID does not exist" Jun 12 18:44:24.201751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201728 2577 scope.go:117] "RemoveContainer" containerID="6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" Jun 12 18:44:24.201965 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.201947 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926\": container with ID starting with 6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926 not found: ID does not exist" containerID="6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" Jun 12 18:44:24.202003 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201971 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926"} err="failed to get container status \"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926\": rpc error: code = NotFound desc = could not find container \"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926\": container with ID starting with 6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926 not found: ID does not exist" Jun 12 18:44:24.202003 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.201985 2577 scope.go:117] "RemoveContainer" containerID="6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" Jun 12 18:44:24.202242 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.202225 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830\": container with ID starting with 6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830 not found: ID does not exist" containerID="6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" Jun 12 18:44:24.202300 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202244 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830"} err="failed to get container status \"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830\": rpc error: code = NotFound desc = could not find container \"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830\": container with ID starting with 6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830 not found: ID does not exist" Jun 12 18:44:24.202300 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202258 2577 scope.go:117] "RemoveContainer" containerID="60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" Jun 12 18:44:24.202493 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.202475 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54\": container with ID starting with 60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54 not found: ID does not exist" containerID="60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" Jun 12 18:44:24.202613 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202501 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54"} err="failed to get container status \"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54\": rpc error: code = NotFound desc = could not find container \"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54\": container with ID starting with 60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54 not found: ID does not exist" Jun 12 18:44:24.202613 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202522 2577 scope.go:117] "RemoveContainer" containerID="f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" Jun 12 18:44:24.202814 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.202797 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8\": container with ID starting with f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8 not found: ID does not exist" containerID="f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" Jun 12 18:44:24.202867 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202825 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8"} err="failed to get container status \"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8\": rpc error: code = NotFound desc = could not find container \"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8\": container with ID starting with f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8 not found: ID does not exist" Jun 12 18:44:24.202867 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.202847 2577 scope.go:117] "RemoveContainer" containerID="ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5" Jun 12 18:44:24.203270 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:44:24.203160 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5\": container with ID starting with ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5 not found: ID does not exist" containerID="ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5" Jun 12 18:44:24.203270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203197 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5"} err="failed to get container status \"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5\": rpc error: code = NotFound desc = could not find container \"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5\": container with ID starting with ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5 not found: ID does not exist" Jun 12 18:44:24.203270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203216 2577 scope.go:117] "RemoveContainer" containerID="7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e" Jun 12 18:44:24.203537 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203493 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e"} err="failed to get container status \"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e\": rpc error: code = NotFound desc = could not find container \"7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e\": container with ID starting with 7ff0e55a825204e3819937454df8062de7ba65b26fb9c2e40b87ad71dcb3fd9e not found: ID does not exist" Jun 12 18:44:24.203537 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203515 2577 scope.go:117] "RemoveContainer" containerID="570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7" Jun 12 18:44:24.203830 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203810 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7"} err="failed to get container status \"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7\": rpc error: code = NotFound desc = could not find container \"570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7\": container with ID starting with 570f5d3d407b21cb713651bf7a8d6b1296ffd01a8417d5315323c7a96549cfb7 not found: ID does not exist" Jun 12 18:44:24.203911 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.203830 2577 scope.go:117] "RemoveContainer" containerID="6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926" Jun 12 18:44:24.204076 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204059 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926"} err="failed to get container status \"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926\": rpc error: code = NotFound desc = could not find container \"6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926\": container with ID starting with 6583834eb360ba7d40cedf475af11678850497ab9d81e635ad6e0fd7bf079926 not found: ID does not exist" Jun 12 18:44:24.204147 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204077 2577 scope.go:117] "RemoveContainer" containerID="6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830" Jun 12 18:44:24.204328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204290 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830"} err="failed to get container status \"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830\": rpc error: code = NotFound desc = could not find container \"6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830\": container with ID starting with 6be1a0479a329931ae2dd90d59f9bfa304e55ca3f23f365de4b07219d4a06830 not found: ID does not exist" Jun 12 18:44:24.204416 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204330 2577 scope.go:117] "RemoveContainer" containerID="60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54" Jun 12 18:44:24.204547 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204522 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54"} err="failed to get container status \"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54\": rpc error: code = NotFound desc = could not find container \"60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54\": container with ID starting with 60eab2bfe61dbe79c85cdacaa247794194be2602d2c388d59fc1afe07f96cc54 not found: ID does not exist" Jun 12 18:44:24.204602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204550 2577 scope.go:117] "RemoveContainer" containerID="f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8" Jun 12 18:44:24.204801 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204770 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8"} err="failed to get container status \"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8\": rpc error: code = NotFound desc = could not find container \"f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8\": container with ID starting with f5320962b9d9f97a6660fa367f08f4bf7fea46bcbb3e2093a82af12d54d17ac8 not found: ID does not exist" Jun 12 18:44:24.204801 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204802 2577 scope.go:117] "RemoveContainer" containerID="ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5" Jun 12 18:44:24.205008 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.204990 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:24.205050 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.205005 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5"} err="failed to get container status \"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5\": rpc error: code = NotFound desc = could not find container \"ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5\": container with ID starting with ed1405ce72049991f16ec4f4f2bd478b0c5aa9e5fb5ae6d59c447c96627f75b5 not found: ID does not exist" Jun 12 18:44:24.246688 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246635 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:24.246892 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246879 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-web" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246892 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-web" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246902 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" containerName="console" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246908 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" containerName="console" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246919 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-metric" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246925 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-metric" Jun 12 18:44:24.246931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246931 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246937 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246943 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" containerName="registry" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246949 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" containerName="registry" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246961 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="prom-label-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246966 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="prom-label-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246973 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246978 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246984 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834205ae-ff46-435d-83ae-63db35ea33ff" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246988 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="834205ae-ff46-435d-83ae-63db35ea33ff" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246994 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="alertmanager" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.246999 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="alertmanager" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247005 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="init-config-reloader" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247009 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="init-config-reloader" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247016 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="config-reloader" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247021 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="config-reloader" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247064 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf76791-4b62-428c-bb0a-71f7aff8fde2" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247070 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="prom-label-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247075 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="834205ae-ff46-435d-83ae-63db35ea33ff" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247081 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbdf4c18-6294-45fd-aafd-3794eeac1f5e" containerName="console" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247087 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2eaf655-5707-4380-996e-15531eecd4a3" containerName="registry" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247093 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-web" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247099 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="alertmanager" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247104 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247108 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="kube-rbac-proxy-metric" Jun 12 18:44:24.247134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.247114 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" containerName="config-reloader" Jun 12 18:44:24.251924 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.251906 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.254668 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.254650 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 12 18:44:24.257809 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.257791 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 12 18:44:24.257908 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.257805 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 12 18:44:24.258004 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.257986 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wj7mv\"" Jun 12 18:44:24.258240 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.258225 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 12 18:44:24.258305 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.258238 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 12 18:44:24.259036 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.258535 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 12 18:44:24.259036 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.258535 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 12 18:44:24.259036 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.258803 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 12 18:44:24.264759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.264741 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 12 18:44:24.270022 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.270002 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:24.320975 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.320944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321090 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.320994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321090 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-web-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27xtq\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-kube-api-access-27xtq\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321251 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-out\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.321442 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.321280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422178 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422178 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422178 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422302 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422302 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-web-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27xtq\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-kube-api-access-27xtq\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422380 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-out\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422492 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.422759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.422570 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.423083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.423059 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.424011 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.423986 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.424966 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.424926 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425105 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425081 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425269 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425240 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425396 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425266 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425464 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425564 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.425638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.425622 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.426370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.426346 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-config-out\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.426622 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.426605 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-web-config\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.427065 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.427043 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.432554 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.432533 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27xtq\" (UniqueName: \"kubernetes.io/projected/04adeeed-3bb7-4f07-8713-c0b0e9f711aa-kube-api-access-27xtq\") pod \"alertmanager-main-0\" (UID: \"04adeeed-3bb7-4f07-8713-c0b0e9f711aa\") " pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.561734 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.561702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 12 18:44:24.898909 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.898878 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c633f0a7-cd20-4bf7-8903-d727f15bbdf8" path="/var/lib/kubelet/pods/c633f0a7-cd20-4bf7-8903-d727f15bbdf8/volumes" Jun 12 18:44:24.906527 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:24.906499 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 12 18:44:24.906769 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:44:24.906747 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04adeeed_3bb7_4f07_8713_c0b0e9f711aa.slice/crio-fc34c84c9a86553a647a53466a7b1ab8bf89a631df64da5c0e92f97fed6177b9 WatchSource:0}: Error finding container fc34c84c9a86553a647a53466a7b1ab8bf89a631df64da5c0e92f97fed6177b9: Status 404 returned error can't find the container with id fc34c84c9a86553a647a53466a7b1ab8bf89a631df64da5c0e92f97fed6177b9 Jun 12 18:44:25.158083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:25.157988 2577 generic.go:356] "Generic (PLEG): container finished" podID="04adeeed-3bb7-4f07-8713-c0b0e9f711aa" containerID="d4856f8ceb04fe3905de49fff9919bf7090b19d05279f2d3e17984f3613ce18f" exitCode=0 Jun 12 18:44:25.158083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:25.158030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerDied","Data":"d4856f8ceb04fe3905de49fff9919bf7090b19d05279f2d3e17984f3613ce18f"} Jun 12 18:44:25.158270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:25.158095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"fc34c84c9a86553a647a53466a7b1ab8bf89a631df64da5c0e92f97fed6177b9"} Jun 12 18:44:26.167068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167021 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"642ce96a62aeb417a9e2f57f1cb7606049e4dc507c9539f93a73f8423464303c"} Jun 12 18:44:26.167557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167111 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"ac0ae571952cba30d304ca68867725f315a4f8715475a416d6b563887a044294"} Jun 12 18:44:26.167557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167152 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"334ebe5ec6161620733a59d96ac1c94afb40f12a5fd1bc656d67e3e07235b047"} Jun 12 18:44:26.167557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"80080a0c87c4e3f9c2c9fe55c0057ce4c5324ba92be1b4ae3e87efe5a7f70f20"} Jun 12 18:44:26.167557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167228 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"095eac691b3db707ac8039459547d9495a65fc26c4a9177747cbf5dda49f7eb6"} Jun 12 18:44:26.167557 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.167259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"04adeeed-3bb7-4f07-8713-c0b0e9f711aa","Type":"ContainerStarted","Data":"af06339b1598bd66e3e6aa9a9ba2c76b6a4d44402d41aec597657a7422ef71e8"} Jun 12 18:44:26.202128 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:26.202067 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.185903937 podStartE2EDuration="2.202051551s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.147614ms" isStatefulPod=false podCreationTimestamp="2026-06-12 18:44:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:44:26.199815277 +0000 UTC m=+383.766092512" watchObservedRunningTime="2026-06-12 18:44:26.202051551 +0000 UTC m=+383.768328785" Jun 12 18:44:33.057490 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.057451 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:44:33.061018 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.060999 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.071801 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.071780 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:44:33.199895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.199860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.199895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.199896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.200114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.199956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.200114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.200026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.200114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.200047 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf76s\" (UniqueName: \"kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.200114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.200073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.200114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.200114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301254 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301628 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301628 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301628 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf76s\" (UniqueName: \"kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301628 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.301842 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.301668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.302038 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.302014 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.302273 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.302243 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.302273 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.302259 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.302467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.302348 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.304083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.304062 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.304184 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.304166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.310380 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.310328 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf76s\" (UniqueName: \"kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s\") pod \"console-5b55d6fbd-kh6w5\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.370427 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.370380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:33.711464 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:33.711432 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:44:33.715558 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:44:33.715534 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e5a5b0_e5a2_43c7_97fb_9d11b376a6c3.slice/crio-05279149f0a7103239edd117a37d371ceb6ddf45bfd8dea55ddad3cc954b9ba2 WatchSource:0}: Error finding container 05279149f0a7103239edd117a37d371ceb6ddf45bfd8dea55ddad3cc954b9ba2: Status 404 returned error can't find the container with id 05279149f0a7103239edd117a37d371ceb6ddf45bfd8dea55ddad3cc954b9ba2 Jun 12 18:44:34.200209 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:34.200178 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55d6fbd-kh6w5" event={"ID":"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3","Type":"ContainerStarted","Data":"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105"} Jun 12 18:44:34.200625 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:34.200224 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55d6fbd-kh6w5" event={"ID":"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3","Type":"ContainerStarted","Data":"05279149f0a7103239edd117a37d371ceb6ddf45bfd8dea55ddad3cc954b9ba2"} Jun 12 18:44:34.218698 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:34.218652 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b55d6fbd-kh6w5" podStartSLOduration=1.218632628 podStartE2EDuration="1.218632628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:44:34.218466145 +0000 UTC m=+391.784743389" watchObservedRunningTime="2026-06-12 18:44:34.218632628 +0000 UTC m=+391.784909865" Jun 12 18:44:43.371118 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:43.371080 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:43.371513 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:43.371143 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:43.375950 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:43.375925 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:44.241465 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:44.241430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:44:44.289610 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:44:44.289583 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:45:09.321126 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.321085 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cd59b76b7-t2zfr" podUID="192bbea2-dcfc-4df5-baea-366c34eb9043" containerName="console" containerID="cri-o://37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664" gracePeriod=15 Jun 12 18:45:09.689168 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.689145 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:45:09.817494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817464 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817508 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817541 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9cp8\" (UniqueName: \"kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817565 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817631 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817660 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.817898 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.817705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle\") pod \"192bbea2-dcfc-4df5-baea-366c34eb9043\" (UID: \"192bbea2-dcfc-4df5-baea-366c34eb9043\") " Jun 12 18:45:09.818110 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.818077 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:45:09.818255 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.818088 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca" (OuterVolumeSpecName: "service-ca") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:45:09.818255 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.818105 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config" (OuterVolumeSpecName: "console-config") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:45:09.818397 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.818296 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:45:09.819777 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.819747 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8" (OuterVolumeSpecName: "kube-api-access-r9cp8") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "kube-api-access-r9cp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:45:09.819874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.819786 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:45:09.819874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.819801 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "192bbea2-dcfc-4df5-baea-366c34eb9043" (UID: "192bbea2-dcfc-4df5-baea-366c34eb9043"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:45:09.918509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918463 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918518 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-console-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918532 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-trusted-ca-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918545 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-service-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918556 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/192bbea2-dcfc-4df5-baea-366c34eb9043-console-oauth-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918567 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9cp8\" (UniqueName: \"kubernetes.io/projected/192bbea2-dcfc-4df5-baea-366c34eb9043-kube-api-access-r9cp8\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:09.918670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:09.918579 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/192bbea2-dcfc-4df5-baea-366c34eb9043-oauth-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:10.334319 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.334281 2577 generic.go:356] "Generic (PLEG): container finished" podID="192bbea2-dcfc-4df5-baea-366c34eb9043" containerID="37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664" exitCode=0 Jun 12 18:45:10.334704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.334361 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd59b76b7-t2zfr" Jun 12 18:45:10.334704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.334364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd59b76b7-t2zfr" event={"ID":"192bbea2-dcfc-4df5-baea-366c34eb9043","Type":"ContainerDied","Data":"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664"} Jun 12 18:45:10.334704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.334440 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd59b76b7-t2zfr" event={"ID":"192bbea2-dcfc-4df5-baea-366c34eb9043","Type":"ContainerDied","Data":"0b37d97f33785a3de096efb97a9a6a63407151446344a8ddb81fbd2d4352f716"} Jun 12 18:45:10.334704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.334460 2577 scope.go:117] "RemoveContainer" containerID="37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664" Jun 12 18:45:10.343263 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.343244 2577 scope.go:117] "RemoveContainer" containerID="37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664" Jun 12 18:45:10.343529 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:45:10.343509 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664\": container with ID starting with 37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664 not found: ID does not exist" containerID="37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664" Jun 12 18:45:10.343598 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.343536 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664"} err="failed to get container status \"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664\": rpc error: code = NotFound desc = could not find container \"37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664\": container with ID starting with 37b3bf9cc62af143ccd42ade1c5b99f7f04fcde95afa0f92980b855861b84664 not found: ID does not exist" Jun 12 18:45:10.357776 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.357750 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:45:10.361037 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.361015 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd59b76b7-t2zfr"] Jun 12 18:45:10.898908 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:10.898877 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192bbea2-dcfc-4df5-baea-366c34eb9043" path="/var/lib/kubelet/pods/192bbea2-dcfc-4df5-baea-366c34eb9043/volumes" Jun 12 18:45:17.351970 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.351936 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc"] Jun 12 18:45:17.352375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.352218 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192bbea2-dcfc-4df5-baea-366c34eb9043" containerName="console" Jun 12 18:45:17.352375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.352229 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="192bbea2-dcfc-4df5-baea-366c34eb9043" containerName="console" Jun 12 18:45:17.352375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.352282 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="192bbea2-dcfc-4df5-baea-366c34eb9043" containerName="console" Jun 12 18:45:17.356670 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.356652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.360402 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.360379 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:45:17.360535 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.360423 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:45:17.360535 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.360508 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:45:17.372210 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.372188 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc"] Jun 12 18:45:17.484029 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.484000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.484238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.484057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksr2g\" (UniqueName: \"kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.484238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.484143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.585512 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.585474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.585709 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.585542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksr2g\" (UniqueName: \"kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.585709 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.585573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.585931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.585906 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.585996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.585914 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.595285 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.595263 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksr2g\" (UniqueName: \"kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:17.666976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:17.666940 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:18.000637 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:18.000502 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc"] Jun 12 18:45:18.002885 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:45:18.002854 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78ada99_0c10_4d04_a3e5_5edea5961fcc.slice/crio-02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d WatchSource:0}: Error finding container 02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d: Status 404 returned error can't find the container with id 02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d Jun 12 18:45:18.365975 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:18.365890 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerStarted","Data":"02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d"} Jun 12 18:45:25.397619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:25.397581 2577 generic.go:356] "Generic (PLEG): container finished" podID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerID="0b5624e3decc6c64842c3c1df5deb4c78096d5ca0506c75303f12471f67ee9b8" exitCode=0 Jun 12 18:45:25.398000 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:25.397660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerDied","Data":"0b5624e3decc6c64842c3c1df5deb4c78096d5ca0506c75303f12471f67ee9b8"} Jun 12 18:45:27.408804 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:27.408773 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerStarted","Data":"112fa0cff8ae8609f0d8376a3344d4c94cfe6f44a556da7bd212ff0eb0e3fba2"} Jun 12 18:45:28.414433 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:28.414400 2577 generic.go:356] "Generic (PLEG): container finished" podID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerID="112fa0cff8ae8609f0d8376a3344d4c94cfe6f44a556da7bd212ff0eb0e3fba2" exitCode=0 Jun 12 18:45:28.414807 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:28.414490 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerDied","Data":"112fa0cff8ae8609f0d8376a3344d4c94cfe6f44a556da7bd212ff0eb0e3fba2"} Jun 12 18:45:37.454088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:37.454050 2577 generic.go:356] "Generic (PLEG): container finished" podID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerID="dccce5574c308d075e950ed69b283b087facc5a090c43cd57a579d631776b3d8" exitCode=0 Jun 12 18:45:37.454505 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:37.454117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerDied","Data":"dccce5574c308d075e950ed69b283b087facc5a090c43cd57a579d631776b3d8"} Jun 12 18:45:38.680906 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.680879 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:38.767468 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.767382 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksr2g\" (UniqueName: \"kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g\") pod \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " Jun 12 18:45:38.767468 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.767420 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle\") pod \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " Jun 12 18:45:38.767468 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.767462 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util\") pod \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\" (UID: \"d78ada99-0c10-4d04-a3e5-5edea5961fcc\") " Jun 12 18:45:38.768104 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.768066 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle" (OuterVolumeSpecName: "bundle") pod "d78ada99-0c10-4d04-a3e5-5edea5961fcc" (UID: "d78ada99-0c10-4d04-a3e5-5edea5961fcc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:45:38.769638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.769607 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g" (OuterVolumeSpecName: "kube-api-access-ksr2g") pod "d78ada99-0c10-4d04-a3e5-5edea5961fcc" (UID: "d78ada99-0c10-4d04-a3e5-5edea5961fcc"). InnerVolumeSpecName "kube-api-access-ksr2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:45:38.771730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.771710 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util" (OuterVolumeSpecName: "util") pod "d78ada99-0c10-4d04-a3e5-5edea5961fcc" (UID: "d78ada99-0c10-4d04-a3e5-5edea5961fcc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:45:38.868985 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.868947 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:38.868985 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.868979 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksr2g\" (UniqueName: \"kubernetes.io/projected/d78ada99-0c10-4d04-a3e5-5edea5961fcc-kube-api-access-ksr2g\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:38.868985 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:38.868990 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d78ada99-0c10-4d04-a3e5-5edea5961fcc-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:39.465100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:39.465063 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" event={"ID":"d78ada99-0c10-4d04-a3e5-5edea5961fcc","Type":"ContainerDied","Data":"02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d"} Jun 12 18:45:39.465100 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:39.465098 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqgdcc" Jun 12 18:45:39.465306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:39.465141 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02c719203ea5dad8f15961ad19c9c8e86d1ca086c506ae8ad45cfee3adc6e67d" Jun 12 18:45:45.591855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.591822 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2"] Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592129 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="util" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592141 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="util" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592155 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="pull" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592160 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="pull" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592167 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="extract" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592173 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="extract" Jun 12 18:45:45.592238 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.592222 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78ada99-0c10-4d04-a3e5-5edea5961fcc" containerName="extract" Jun 12 18:45:45.596848 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.596830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.599494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.599470 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:45:45.599494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.599485 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-whkh7\"" Jun 12 18:45:45.599652 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.599515 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 12 18:45:45.609978 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.609952 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2"] Jun 12 18:45:45.623467 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.623441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2e09696-6bee-402a-adfe-cad0bd7ec915-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.623562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.623501 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gh2k\" (UniqueName: \"kubernetes.io/projected/a2e09696-6bee-402a-adfe-cad0bd7ec915-kube-api-access-5gh2k\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.724279 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.724249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2e09696-6bee-402a-adfe-cad0bd7ec915-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.724448 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.724322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gh2k\" (UniqueName: \"kubernetes.io/projected/a2e09696-6bee-402a-adfe-cad0bd7ec915-kube-api-access-5gh2k\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.724602 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.724583 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2e09696-6bee-402a-adfe-cad0bd7ec915-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.733771 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.733669 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gh2k\" (UniqueName: \"kubernetes.io/projected/a2e09696-6bee-402a-adfe-cad0bd7ec915-kube-api-access-5gh2k\") pod \"cert-manager-operator-controller-manager-57757f7cfc-j2wj2\" (UID: \"a2e09696-6bee-402a-adfe-cad0bd7ec915\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:45.905349 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:45.905302 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" Jun 12 18:45:46.251780 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:46.251754 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2"] Jun 12 18:45:46.253940 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:45:46.253904 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e09696_6bee_402a_adfe_cad0bd7ec915.slice/crio-b6ea7e0c9c27e40bb95a00544705dad784ec5e3219819040d693a80e8bf33fea WatchSource:0}: Error finding container b6ea7e0c9c27e40bb95a00544705dad784ec5e3219819040d693a80e8bf33fea: Status 404 returned error can't find the container with id b6ea7e0c9c27e40bb95a00544705dad784ec5e3219819040d693a80e8bf33fea Jun 12 18:45:46.493638 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:46.493601 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" event={"ID":"a2e09696-6bee-402a-adfe-cad0bd7ec915","Type":"ContainerStarted","Data":"b6ea7e0c9c27e40bb95a00544705dad784ec5e3219819040d693a80e8bf33fea"} Jun 12 18:45:48.505023 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:48.504988 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" event={"ID":"a2e09696-6bee-402a-adfe-cad0bd7ec915","Type":"ContainerStarted","Data":"f0fee6a692380b4fc417195fde555d6b2ac63722f3bf2433f879d05dafa7681e"} Jun 12 18:45:48.527736 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:48.527684 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-j2wj2" podStartSLOduration=2.2044897629999998 podStartE2EDuration="3.527669011s" totalImagesPullingTime="1.323179248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:45:48.526153067 +0000 UTC m=+466.092430301" watchObservedRunningTime="2026-06-12 18:45:48.527669011 +0000 UTC m=+466.093946245" Jun 12 18:45:50.057332 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.054288 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l"] Jun 12 18:45:50.058303 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.058283 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.063089 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.063066 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:45:50.063293 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.063279 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:45:50.063749 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.063730 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:45:50.068776 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.068753 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l"] Jun 12 18:45:50.112576 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.112543 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv"] Jun 12 18:45:50.114694 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.114679 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.117088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.117068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 12 18:45:50.117206 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.117189 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 12 18:45:50.127576 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.127556 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv"] Jun 12 18:45:50.164254 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.164222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.164460 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.164280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmtnf\" (UniqueName: \"kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.164460 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.164382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.164460 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.164424 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.164608 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.164477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc42p\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-kube-api-access-vc42p\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.265681 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.265643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmtnf\" (UniqueName: \"kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.265874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.265694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.265874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.265715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.265874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.265750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc42p\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-kube-api-access-vc42p\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.265874 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.265796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.266181 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.266160 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.266243 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.266194 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.277625 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.277600 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.277723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.277659 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmtnf\" (UniqueName: \"kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.277879 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.277857 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc42p\" (UniqueName: \"kubernetes.io/projected/46efef82-97d7-47d5-8244-b0a4237e8fa2-kube-api-access-vc42p\") pod \"cert-manager-webhook-7dc76cbb94-qnhpv\" (UID: \"46efef82-97d7-47d5-8244-b0a4237e8fa2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.367591 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.367504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:50.423598 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.423564 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:50.706466 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.706444 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l"] Jun 12 18:45:50.708847 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:45:50.708825 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf15804_9b67_4eea_ab01_4bc17aa2c0bd.slice/crio-844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57 WatchSource:0}: Error finding container 844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57: Status 404 returned error can't find the container with id 844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57 Jun 12 18:45:50.765887 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:50.765772 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv"] Jun 12 18:45:50.768251 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:45:50.768227 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46efef82_97d7_47d5_8244_b0a4237e8fa2.slice/crio-2e34b5954f4cb77beabf435c5f901bb7630c6a267ecb3717c9fdb3142ba12e36 WatchSource:0}: Error finding container 2e34b5954f4cb77beabf435c5f901bb7630c6a267ecb3717c9fdb3142ba12e36: Status 404 returned error can't find the container with id 2e34b5954f4cb77beabf435c5f901bb7630c6a267ecb3717c9fdb3142ba12e36 Jun 12 18:45:51.520328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:51.520280 2577 generic.go:356] "Generic (PLEG): container finished" podID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerID="2b323509c5af231e27fb5eca96637b7260ef29c1b74009a021d7285d1ff5df2f" exitCode=0 Jun 12 18:45:51.520791 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:51.520383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" event={"ID":"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd","Type":"ContainerDied","Data":"2b323509c5af231e27fb5eca96637b7260ef29c1b74009a021d7285d1ff5df2f"} Jun 12 18:45:51.520791 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:51.520458 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" event={"ID":"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd","Type":"ContainerStarted","Data":"844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57"} Jun 12 18:45:51.521739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:51.521709 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" event={"ID":"46efef82-97d7-47d5-8244-b0a4237e8fa2","Type":"ContainerStarted","Data":"2e34b5954f4cb77beabf435c5f901bb7630c6a267ecb3717c9fdb3142ba12e36"} Jun 12 18:45:53.535248 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.535185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" event={"ID":"46efef82-97d7-47d5-8244-b0a4237e8fa2","Type":"ContainerStarted","Data":"6b26d194af84e175b0a83b1b3d81551856e808bbf7dd3f5bc877bc654b644f9a"} Jun 12 18:45:53.535701 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.535357 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:45:53.553428 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.553378 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" podStartSLOduration=1.334089099 podStartE2EDuration="3.553359742s" totalImagesPullingTime="2.219270643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:45:53.552262206 +0000 UTC m=+471.118539454" watchObservedRunningTime="2026-06-12 18:45:53.553359742 +0000 UTC m=+471.119636975" Jun 12 18:45:53.853904 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.853823 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj"] Jun 12 18:45:53.858481 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.858462 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:53.861212 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.861188 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7bskc\"" Jun 12 18:45:53.866819 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.866785 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj"] Jun 12 18:45:53.895813 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.895778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:53.896000 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.895893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9qhm\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-kube-api-access-t9qhm\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:53.996843 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.996804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9qhm\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-kube-api-access-t9qhm\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:53.997234 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:53.997205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:54.005912 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:54.005888 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:54.005912 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:54.005896 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9qhm\" (UniqueName: \"kubernetes.io/projected/ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01-kube-api-access-t9qhm\") pod \"cert-manager-cainjector-5d99449ffd-jmqnj\" (UID: \"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:54.169662 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:54.169625 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" Jun 12 18:45:54.856868 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:54.856844 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj"] Jun 12 18:45:54.859086 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:45:54.859060 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podced7d2d2_d4c2_4945_8bbf_71fbe8c38a01.slice/crio-f4d549ff185b0d81aaa20bdeb95cc103a34951c0f5fb65f38cfeeaf2a3dad9a6 WatchSource:0}: Error finding container f4d549ff185b0d81aaa20bdeb95cc103a34951c0f5fb65f38cfeeaf2a3dad9a6: Status 404 returned error can't find the container with id f4d549ff185b0d81aaa20bdeb95cc103a34951c0f5fb65f38cfeeaf2a3dad9a6 Jun 12 18:45:55.547539 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:55.547503 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" event={"ID":"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01","Type":"ContainerStarted","Data":"381d42525eb3998f37e00b48f0172cc83674ab04331aa91052f0dc164a694415"} Jun 12 18:45:55.547539 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:55.547548 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" event={"ID":"ced7d2d2-d4c2-4945-8bbf-71fbe8c38a01","Type":"ContainerStarted","Data":"f4d549ff185b0d81aaa20bdeb95cc103a34951c0f5fb65f38cfeeaf2a3dad9a6"} Jun 12 18:45:55.549205 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:55.549182 2577 generic.go:356] "Generic (PLEG): container finished" podID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerID="bca9d3ce4a45d28a8653f9fa751f5c2c6259f00e6f51b086460fb92d8504e35b" exitCode=0 Jun 12 18:45:55.549349 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:55.549231 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" event={"ID":"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd","Type":"ContainerDied","Data":"bca9d3ce4a45d28a8653f9fa751f5c2c6259f00e6f51b086460fb92d8504e35b"} Jun 12 18:45:55.563680 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:55.563639 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-jmqnj" podStartSLOduration=2.563626052 podStartE2EDuration="2.563626052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:45:55.562565468 +0000 UTC m=+473.128842719" watchObservedRunningTime="2026-06-12 18:45:55.563626052 +0000 UTC m=+473.129903287" Jun 12 18:45:56.555992 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:56.555953 2577 generic.go:356] "Generic (PLEG): container finished" podID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerID="503eb441182351bdf14f2481397512a95fad7875bb84ffeb46ad01469b29f69e" exitCode=0 Jun 12 18:45:56.556363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:56.556045 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" event={"ID":"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd","Type":"ContainerDied","Data":"503eb441182351bdf14f2481397512a95fad7875bb84ffeb46ad01469b29f69e"} Jun 12 18:45:57.796849 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.796823 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:57.931074 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.931043 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util\") pod \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " Jun 12 18:45:57.931285 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.931084 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle\") pod \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " Jun 12 18:45:57.931285 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.931157 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmtnf\" (UniqueName: \"kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf\") pod \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\" (UID: \"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd\") " Jun 12 18:45:57.931561 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.931528 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle" (OuterVolumeSpecName: "bundle") pod "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" (UID: "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:45:57.933166 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.933142 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf" (OuterVolumeSpecName: "kube-api-access-qmtnf") pod "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" (UID: "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd"). InnerVolumeSpecName "kube-api-access-qmtnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:45:57.936159 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:57.936135 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util" (OuterVolumeSpecName: "util") pod "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" (UID: "0cf15804-9b67-4eea-ab01-4bc17aa2c0bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:45:58.032474 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.032443 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:58.032474 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.032467 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:58.032474 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.032476 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmtnf\" (UniqueName: \"kubernetes.io/projected/0cf15804-9b67-4eea-ab01-4bc17aa2c0bd-kube-api-access-qmtnf\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:45:58.565455 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.565415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" event={"ID":"0cf15804-9b67-4eea-ab01-4bc17aa2c0bd","Type":"ContainerDied","Data":"844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57"} Jun 12 18:45:58.565601 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.565466 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="844373b0b825552113a4b8178401d58cb17f9bc5ec11d9107af60d186f05ff57" Jun 12 18:45:58.565601 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:58.565435 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfn2l" Jun 12 18:45:59.543891 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:45:59.543862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qnhpv" Jun 12 18:46:06.587002 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.586964 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp"] Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587475 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="pull" Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587496 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="pull" Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587512 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="util" Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587520 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="util" Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587533 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="extract" Jun 12 18:46:06.587562 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587541 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="extract" Jun 12 18:46:06.587853 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.587615 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf15804-9b67-4eea-ab01-4bc17aa2c0bd" containerName="extract" Jun 12 18:46:06.640499 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.640461 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp"] Jun 12 18:46:06.640645 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.640550 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.643401 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.643369 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 12 18:46:06.643526 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.643414 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 12 18:46:06.644444 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.644427 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4vslw\"" Jun 12 18:46:06.701579 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.701555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae2816e3-d02e-49f0-ba61-3d3e804c6631-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.701685 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.701615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdg9n\" (UniqueName: \"kubernetes.io/projected/ae2816e3-d02e-49f0-ba61-3d3e804c6631-kube-api-access-gdg9n\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.802430 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.802400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae2816e3-d02e-49f0-ba61-3d3e804c6631-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.802556 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.802479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg9n\" (UniqueName: \"kubernetes.io/projected/ae2816e3-d02e-49f0-ba61-3d3e804c6631-kube-api-access-gdg9n\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.802756 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.802738 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae2816e3-d02e-49f0-ba61-3d3e804c6631-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.811723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.811693 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdg9n\" (UniqueName: \"kubernetes.io/projected/ae2816e3-d02e-49f0-ba61-3d3e804c6631-kube-api-access-gdg9n\") pod \"openshift-lws-operator-fd8ccff4c-zmttp\" (UID: \"ae2816e3-d02e-49f0-ba61-3d3e804c6631\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:06.949371 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:06.949339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" Jun 12 18:46:07.302851 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:07.302823 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp"] Jun 12 18:46:07.305687 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:46:07.305657 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae2816e3_d02e_49f0_ba61_3d3e804c6631.slice/crio-ffaddc3a31329efa760af35253f6fb96e9d57cbf25c3b5274bd78f6d34ccba17 WatchSource:0}: Error finding container ffaddc3a31329efa760af35253f6fb96e9d57cbf25c3b5274bd78f6d34ccba17: Status 404 returned error can't find the container with id ffaddc3a31329efa760af35253f6fb96e9d57cbf25c3b5274bd78f6d34ccba17 Jun 12 18:46:07.608083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:07.608005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" event={"ID":"ae2816e3-d02e-49f0-ba61-3d3e804c6631","Type":"ContainerStarted","Data":"ffaddc3a31329efa760af35253f6fb96e9d57cbf25c3b5274bd78f6d34ccba17"} Jun 12 18:46:09.618782 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:09.618684 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" event={"ID":"ae2816e3-d02e-49f0-ba61-3d3e804c6631","Type":"ContainerStarted","Data":"51ce985eae15bc18928db8a9b1a8411b75cbe66b89799fc35f57c354c3279bd8"} Jun 12 18:46:09.637762 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:09.637714 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zmttp" podStartSLOduration=1.5951914459999998 podStartE2EDuration="3.637699411s" totalImagesPullingTime="2.042507965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:46:09.635774243 +0000 UTC m=+487.202051497" watchObservedRunningTime="2026-06-12 18:46:09.637699411 +0000 UTC m=+487.203976644" Jun 12 18:46:14.064536 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.064497 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr"] Jun 12 18:46:14.067016 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.066996 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.070382 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.070358 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:46:14.071537 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.071520 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:46:14.071537 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.071531 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:46:14.082030 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.082002 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr"] Jun 12 18:46:14.167375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.167331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.167375 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.167380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.167583 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.167415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr29n\" (UniqueName: \"kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.268299 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.268263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.268501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.268333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.268501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.268373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr29n\" (UniqueName: \"kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.268679 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.268660 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.268714 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.268689 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.277968 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.277940 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr29n\" (UniqueName: \"kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.375906 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.375815 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:14.709704 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:14.709582 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr"] Jun 12 18:46:14.712409 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:46:14.712384 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd718bb46_eb40_4d89_9e41_dfdeeb132f14.slice/crio-0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf WatchSource:0}: Error finding container 0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf: Status 404 returned error can't find the container with id 0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf Jun 12 18:46:15.649549 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:15.649516 2577 generic.go:356] "Generic (PLEG): container finished" podID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerID="a5c53fd11d7a607b7d4407cf29fe20aee9c6bb4f79251982898576b6baadc090" exitCode=0 Jun 12 18:46:15.649931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:15.649605 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" event={"ID":"d718bb46-eb40-4d89-9e41-dfdeeb132f14","Type":"ContainerDied","Data":"a5c53fd11d7a607b7d4407cf29fe20aee9c6bb4f79251982898576b6baadc090"} Jun 12 18:46:15.649931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:15.649656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" event={"ID":"d718bb46-eb40-4d89-9e41-dfdeeb132f14","Type":"ContainerStarted","Data":"0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf"} Jun 12 18:46:16.655552 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:16.655519 2577 generic.go:356] "Generic (PLEG): container finished" podID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerID="aeaeb6fc9680b5f002a9438f20a035fc4f88eb3750f24850fcbc1710346812b6" exitCode=0 Jun 12 18:46:16.655916 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:16.655583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" event={"ID":"d718bb46-eb40-4d89-9e41-dfdeeb132f14","Type":"ContainerDied","Data":"aeaeb6fc9680b5f002a9438f20a035fc4f88eb3750f24850fcbc1710346812b6"} Jun 12 18:46:17.661936 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:17.661906 2577 generic.go:356] "Generic (PLEG): container finished" podID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerID="a6accb3d3d4ab37ca777a9595cece0e7dd517a304855b1ed2b167be150de25a3" exitCode=0 Jun 12 18:46:17.662299 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:17.661996 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" event={"ID":"d718bb46-eb40-4d89-9e41-dfdeeb132f14","Type":"ContainerDied","Data":"a6accb3d3d4ab37ca777a9595cece0e7dd517a304855b1ed2b167be150de25a3"} Jun 12 18:46:18.903199 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:18.903168 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:19.008820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.008736 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util\") pod \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " Jun 12 18:46:19.008820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.008781 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr29n\" (UniqueName: \"kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n\") pod \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " Jun 12 18:46:19.009002 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.008834 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle\") pod \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\" (UID: \"d718bb46-eb40-4d89-9e41-dfdeeb132f14\") " Jun 12 18:46:19.009723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.009690 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle" (OuterVolumeSpecName: "bundle") pod "d718bb46-eb40-4d89-9e41-dfdeeb132f14" (UID: "d718bb46-eb40-4d89-9e41-dfdeeb132f14"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:46:19.010800 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.010777 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n" (OuterVolumeSpecName: "kube-api-access-hr29n") pod "d718bb46-eb40-4d89-9e41-dfdeeb132f14" (UID: "d718bb46-eb40-4d89-9e41-dfdeeb132f14"). InnerVolumeSpecName "kube-api-access-hr29n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:46:19.014627 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.014590 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util" (OuterVolumeSpecName: "util") pod "d718bb46-eb40-4d89-9e41-dfdeeb132f14" (UID: "d718bb46-eb40-4d89-9e41-dfdeeb132f14"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:46:19.109863 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.109839 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:19.109863 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.109862 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr29n\" (UniqueName: \"kubernetes.io/projected/d718bb46-eb40-4d89-9e41-dfdeeb132f14-kube-api-access-hr29n\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:19.110001 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.109873 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d718bb46-eb40-4d89-9e41-dfdeeb132f14-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:19.672338 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.672295 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" Jun 12 18:46:19.672524 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.672343 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdrbgjr" event={"ID":"d718bb46-eb40-4d89-9e41-dfdeeb132f14","Type":"ContainerDied","Data":"0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf"} Jun 12 18:46:19.672524 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:19.672387 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0949ad28ee812fde78a4ca81fc5182d1f3570c36b1889785836f226950adfaaf" Jun 12 18:46:25.910972 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.910933 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6"] Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911264 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="util" Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911277 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="util" Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911289 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="extract" Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911294 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="extract" Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911304 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="pull" Jun 12 18:46:25.911372 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911333 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="pull" Jun 12 18:46:25.911564 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.911386 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d718bb46-eb40-4d89-9e41-dfdeeb132f14" containerName="extract" Jun 12 18:46:25.913631 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.913612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:25.919328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.919287 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 12 18:46:25.919471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.919298 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 12 18:46:25.919586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.919386 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 12 18:46:25.919586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.919400 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 12 18:46:25.920844 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.920820 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m6wb2\"" Jun 12 18:46:25.941731 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:25.941708 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6"] Jun 12 18:46:26.070104 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.070066 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-tmp\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.070306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.070109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.070306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.070134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr9kw\" (UniqueName: \"kubernetes.io/projected/f797a377-c317-482b-8fd3-2492eb270c16-kube-api-access-zr9kw\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.070306 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.070272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-manifests\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.070483 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.070333 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-manifests\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-tmp\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171440 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr9kw\" (UniqueName: \"kubernetes.io/projected/f797a377-c317-482b-8fd3-2492eb270c16-kube-api-access-zr9kw\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.171531 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.171509 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-manifests\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.173494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.173463 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797a377-c317-482b-8fd3-2492eb270c16-tmp\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.173606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.173497 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.173606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.173597 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f797a377-c317-482b-8fd3-2492eb270c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.183360 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.183339 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr9kw\" (UniqueName: \"kubernetes.io/projected/f797a377-c317-482b-8fd3-2492eb270c16-kube-api-access-zr9kw\") pod \"opendatahub-operator-controller-manager-bc6c5cc78-fxsm6\" (UID: \"f797a377-c317-482b-8fd3-2492eb270c16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.223924 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.223900 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:26.583820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.583737 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6"] Jun 12 18:46:26.587743 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:46:26.587720 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf797a377_c317_482b_8fd3_2492eb270c16.slice/crio-ef28f451afc57b46dfe7d426111734bd6b2fb57b81f5c2597753f3f1642978e6 WatchSource:0}: Error finding container ef28f451afc57b46dfe7d426111734bd6b2fb57b81f5c2597753f3f1642978e6: Status 404 returned error can't find the container with id ef28f451afc57b46dfe7d426111734bd6b2fb57b81f5c2597753f3f1642978e6 Jun 12 18:46:26.706947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:26.706912 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" event={"ID":"f797a377-c317-482b-8fd3-2492eb270c16","Type":"ContainerStarted","Data":"ef28f451afc57b46dfe7d426111734bd6b2fb57b81f5c2597753f3f1642978e6"} Jun 12 18:46:29.726989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:29.726954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" event={"ID":"f797a377-c317-482b-8fd3-2492eb270c16","Type":"ContainerStarted","Data":"784572697baeb8c7268faea7a0bb6f71aa40d54b2b580497c599439f388a9492"} Jun 12 18:46:30.738355 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:30.738299 2577 generic.go:356] "Generic (PLEG): container finished" podID="f797a377-c317-482b-8fd3-2492eb270c16" containerID="784572697baeb8c7268faea7a0bb6f71aa40d54b2b580497c599439f388a9492" exitCode=0 Jun 12 18:46:30.738802 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:30.738383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" event={"ID":"f797a377-c317-482b-8fd3-2492eb270c16","Type":"ContainerDied","Data":"784572697baeb8c7268faea7a0bb6f71aa40d54b2b580497c599439f388a9492"} Jun 12 18:46:31.745191 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:31.745156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" event={"ID":"f797a377-c317-482b-8fd3-2492eb270c16","Type":"ContainerStarted","Data":"25e13262df246c4aa0ae054e35506aab475cefeb7b4223f16d95278e34618220"} Jun 12 18:46:31.745588 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:31.745351 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:31.772883 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:31.772838 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" podStartSLOduration=2.743689612 podStartE2EDuration="6.772819541s" totalImagesPullingTime="3.03424831s" totalInitContainerRuntime="994.881619ms" isStatefulPod=false podCreationTimestamp="2026-06-12 18:46:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:46:31.771155934 +0000 UTC m=+509.337433170" watchObservedRunningTime="2026-06-12 18:46:31.772819541 +0000 UTC m=+509.339096777" Jun 12 18:46:42.752669 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:42.752638 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bc6c5cc78-fxsm6" Jun 12 18:46:45.578807 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.578767 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd"] Jun 12 18:46:45.586889 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.586858 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.589738 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.589719 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 12 18:46:45.591183 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.591165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 12 18:46:45.591256 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.591224 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7f76t\"" Jun 12 18:46:45.591303 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.591251 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 12 18:46:45.596178 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.596151 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd"] Jun 12 18:46:45.636012 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.635975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-metrics-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.636190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.636018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74346848-57dd-476a-8d80-27ddce1b1db9-manager-config\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.636190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.636040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.636190 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.636096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wfcs\" (UniqueName: \"kubernetes.io/projected/74346848-57dd-476a-8d80-27ddce1b1db9-kube-api-access-8wfcs\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.736742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.736701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-metrics-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.736742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.736742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74346848-57dd-476a-8d80-27ddce1b1db9-manager-config\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.737010 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.736761 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.737010 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.736789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wfcs\" (UniqueName: \"kubernetes.io/projected/74346848-57dd-476a-8d80-27ddce1b1db9-kube-api-access-8wfcs\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.737475 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.737442 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74346848-57dd-476a-8d80-27ddce1b1db9-manager-config\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.739391 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.739360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.739500 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.739360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74346848-57dd-476a-8d80-27ddce1b1db9-metrics-cert\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.764658 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.764620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wfcs\" (UniqueName: \"kubernetes.io/projected/74346848-57dd-476a-8d80-27ddce1b1db9-kube-api-access-8wfcs\") pod \"lws-controller-manager-576ccf6f5f-825dd\" (UID: \"74346848-57dd-476a-8d80-27ddce1b1db9\") " pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:45.896985 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:45.896952 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:46.253284 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.253255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd"] Jun 12 18:46:46.255128 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:46:46.255103 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74346848_57dd_476a_8d80_27ddce1b1db9.slice/crio-1b9219a3767bb470b41d7fa9896bba828443f398a0e6a114f1bdbfbbf2daf58f WatchSource:0}: Error finding container 1b9219a3767bb470b41d7fa9896bba828443f398a0e6a114f1bdbfbbf2daf58f: Status 404 returned error can't find the container with id 1b9219a3767bb470b41d7fa9896bba828443f398a0e6a114f1bdbfbbf2daf58f Jun 12 18:46:46.816667 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.816631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" event={"ID":"74346848-57dd-476a-8d80-27ddce1b1db9","Type":"ContainerStarted","Data":"1b9219a3767bb470b41d7fa9896bba828443f398a0e6a114f1bdbfbbf2daf58f"} Jun 12 18:46:46.859402 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.859363 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr"] Jun 12 18:46:46.864113 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.864094 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:46.866811 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.866790 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:46:46.867769 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.867752 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:46:46.867769 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.867764 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:46:46.879144 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.876097 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr"] Jun 12 18:46:46.948512 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.948474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:46.948671 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.948519 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh8ns\" (UniqueName: \"kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:46.948671 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:46.948547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.049785 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.049750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.049976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.049789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh8ns\" (UniqueName: \"kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.049976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.049813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.050148 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.050130 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.050217 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.050194 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.060231 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.060201 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh8ns\" (UniqueName: \"kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.175589 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.175539 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:47.526841 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.526781 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr"] Jun 12 18:46:47.529688 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:46:47.529657 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88077c72_0b28_4aa2_b84f_c52411d6a0d2.slice/crio-b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e WatchSource:0}: Error finding container b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e: Status 404 returned error can't find the container with id b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e Jun 12 18:46:47.825866 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.825777 2577 generic.go:356] "Generic (PLEG): container finished" podID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerID="a6b75eaad2f18d6915283ec5492fce728c13f02519b4455c5d92f828b8ceefb3" exitCode=0 Jun 12 18:46:47.826338 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.825859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" event={"ID":"88077c72-0b28-4aa2-b84f-c52411d6a0d2","Type":"ContainerDied","Data":"a6b75eaad2f18d6915283ec5492fce728c13f02519b4455c5d92f828b8ceefb3"} Jun 12 18:46:47.826338 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:47.825917 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" event={"ID":"88077c72-0b28-4aa2-b84f-c52411d6a0d2","Type":"ContainerStarted","Data":"b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e"} Jun 12 18:46:48.831912 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:48.831874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" event={"ID":"74346848-57dd-476a-8d80-27ddce1b1db9","Type":"ContainerStarted","Data":"4d34dee47dd907bfe5fac6b9026b932bbefb08089f93047c58f9cfe1e2e3ec20"} Jun 12 18:46:48.832582 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:48.831942 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:46:48.833998 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:48.833972 2577 generic.go:356] "Generic (PLEG): container finished" podID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerID="d361fde2e4e1023e0a4ed8fcf3a48ab73c0c187d0f8e4aac55308736682dff9c" exitCode=0 Jun 12 18:46:48.834102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:48.834009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" event={"ID":"88077c72-0b28-4aa2-b84f-c52411d6a0d2","Type":"ContainerDied","Data":"d361fde2e4e1023e0a4ed8fcf3a48ab73c0c187d0f8e4aac55308736682dff9c"} Jun 12 18:46:48.854008 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:48.853932 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" podStartSLOduration=1.982621465 podStartE2EDuration="3.853910885s" totalImagesPullingTime="1.87128942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:46:48.849534609 +0000 UTC m=+526.415811844" watchObservedRunningTime="2026-06-12 18:46:48.853910885 +0000 UTC m=+526.420188121" Jun 12 18:46:49.841150 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:49.841112 2577 generic.go:356] "Generic (PLEG): container finished" podID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerID="7576f836f701ea3487aacdb1e61c8026a22d4b1ef4a284b2ec9dcd334429861d" exitCode=0 Jun 12 18:46:49.841544 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:49.841197 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" event={"ID":"88077c72-0b28-4aa2-b84f-c52411d6a0d2","Type":"ContainerDied","Data":"7576f836f701ea3487aacdb1e61c8026a22d4b1ef4a284b2ec9dcd334429861d"} Jun 12 18:46:51.098660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.098596 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:51.185495 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.185460 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh8ns\" (UniqueName: \"kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns\") pod \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " Jun 12 18:46:51.185660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.185510 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util\") pod \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " Jun 12 18:46:51.185660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.185577 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle\") pod \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\" (UID: \"88077c72-0b28-4aa2-b84f-c52411d6a0d2\") " Jun 12 18:46:51.186388 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.186359 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle" (OuterVolumeSpecName: "bundle") pod "88077c72-0b28-4aa2-b84f-c52411d6a0d2" (UID: "88077c72-0b28-4aa2-b84f-c52411d6a0d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:46:51.187626 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.187597 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns" (OuterVolumeSpecName: "kube-api-access-xh8ns") pod "88077c72-0b28-4aa2-b84f-c52411d6a0d2" (UID: "88077c72-0b28-4aa2-b84f-c52411d6a0d2"). InnerVolumeSpecName "kube-api-access-xh8ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:46:51.190855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.190836 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util" (OuterVolumeSpecName: "util") pod "88077c72-0b28-4aa2-b84f-c52411d6a0d2" (UID: "88077c72-0b28-4aa2-b84f-c52411d6a0d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:46:51.286328 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.286286 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:51.286499 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.286341 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh8ns\" (UniqueName: \"kubernetes.io/projected/88077c72-0b28-4aa2-b84f-c52411d6a0d2-kube-api-access-xh8ns\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:51.286499 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.286350 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88077c72-0b28-4aa2-b84f-c52411d6a0d2-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:46:51.852395 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.852302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" event={"ID":"88077c72-0b28-4aa2-b84f-c52411d6a0d2","Type":"ContainerDied","Data":"b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e"} Jun 12 18:46:51.852395 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.852364 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0679hxr" Jun 12 18:46:51.852587 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:51.852371 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2a1aad31c4b76a416abaee66e6705333374b6348434ecbec6588bcaae9dee1e" Jun 12 18:46:59.843329 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:46:59.843281 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-576ccf6f5f-825dd" Jun 12 18:47:01.219646 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219613 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6"] Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219943 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="util" Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219956 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="util" Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219972 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="pull" Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219978 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="pull" Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.219994 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="extract" Jun 12 18:47:01.220006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.220000 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="extract" Jun 12 18:47:01.220205 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.220052 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88077c72-0b28-4aa2-b84f-c52411d6a0d2" containerName="extract" Jun 12 18:47:01.282978 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.282944 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6"] Jun 12 18:47:01.283133 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.283065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.286648 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.286626 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:47:01.286798 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.286738 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:47:01.287729 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.287709 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:47:01.372686 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.372646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.372862 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.372725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48sj6\" (UniqueName: \"kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.372862 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.372753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.473706 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.473623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48sj6\" (UniqueName: \"kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.473706 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.473678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.473954 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.473739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.474134 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.474109 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.474183 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.474118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.509120 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.509067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48sj6\" (UniqueName: \"kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:01.593127 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:01.593090 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:02.019808 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:02.019710 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6"] Jun 12 18:47:02.021950 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:47:02.021928 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd801711e_3c78_4100_878a_f69159662671.slice/crio-ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497 WatchSource:0}: Error finding container ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497: Status 404 returned error can't find the container with id ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497 Jun 12 18:47:02.907022 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:02.906992 2577 generic.go:356] "Generic (PLEG): container finished" podID="d801711e-3c78-4100-878a-f69159662671" containerID="6edd4369951abcad235b188d08823e405f83263747515b413d03b6915989211d" exitCode=0 Jun 12 18:47:02.907501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:02.907074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" event={"ID":"d801711e-3c78-4100-878a-f69159662671","Type":"ContainerDied","Data":"6edd4369951abcad235b188d08823e405f83263747515b413d03b6915989211d"} Jun 12 18:47:02.907501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:02.907117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" event={"ID":"d801711e-3c78-4100-878a-f69159662671","Type":"ContainerStarted","Data":"ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497"} Jun 12 18:47:03.913244 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:03.913211 2577 generic.go:356] "Generic (PLEG): container finished" podID="d801711e-3c78-4100-878a-f69159662671" containerID="e564516ff1fd78c34ec0dc0bb4257a4e2e17f492f2838abf91d758799ca7efef" exitCode=0 Jun 12 18:47:03.913611 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:03.913277 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" event={"ID":"d801711e-3c78-4100-878a-f69159662671","Type":"ContainerDied","Data":"e564516ff1fd78c34ec0dc0bb4257a4e2e17f492f2838abf91d758799ca7efef"} Jun 12 18:47:04.919565 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:04.919531 2577 generic.go:356] "Generic (PLEG): container finished" podID="d801711e-3c78-4100-878a-f69159662671" containerID="cd48920b17f7e28109b7ceb8feb7a8948803dc19b93278b118e7330dbe7f97fa" exitCode=0 Jun 12 18:47:04.920097 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:04.919623 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" event={"ID":"d801711e-3c78-4100-878a-f69159662671","Type":"ContainerDied","Data":"cd48920b17f7e28109b7ceb8feb7a8948803dc19b93278b118e7330dbe7f97fa"} Jun 12 18:47:06.177846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.177790 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:06.317905 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.317803 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48sj6\" (UniqueName: \"kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6\") pod \"d801711e-3c78-4100-878a-f69159662671\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " Jun 12 18:47:06.318098 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.318028 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle\") pod \"d801711e-3c78-4100-878a-f69159662671\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " Jun 12 18:47:06.318176 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.318114 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util\") pod \"d801711e-3c78-4100-878a-f69159662671\" (UID: \"d801711e-3c78-4100-878a-f69159662671\") " Jun 12 18:47:06.318882 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.318857 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle" (OuterVolumeSpecName: "bundle") pod "d801711e-3c78-4100-878a-f69159662671" (UID: "d801711e-3c78-4100-878a-f69159662671"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:47:06.319855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.319824 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6" (OuterVolumeSpecName: "kube-api-access-48sj6") pod "d801711e-3c78-4100-878a-f69159662671" (UID: "d801711e-3c78-4100-878a-f69159662671"). InnerVolumeSpecName "kube-api-access-48sj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:47:06.323809 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.323771 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util" (OuterVolumeSpecName: "util") pod "d801711e-3c78-4100-878a-f69159662671" (UID: "d801711e-3c78-4100-878a-f69159662671"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:47:06.418893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.418856 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:47:06.418893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.418885 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d801711e-3c78-4100-878a-f69159662671-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:47:06.418893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.418896 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48sj6\" (UniqueName: \"kubernetes.io/projected/d801711e-3c78-4100-878a-f69159662671-kube-api-access-48sj6\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:47:06.931563 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.931537 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" Jun 12 18:47:06.931705 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.931565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a9r5s6" event={"ID":"d801711e-3c78-4100-878a-f69159662671","Type":"ContainerDied","Data":"ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497"} Jun 12 18:47:06.931705 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:06.931601 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac30dcb813449cdbbaea1348d969dcb0526b3534bc4139618b603ecdbf36f497" Jun 12 18:47:21.155218 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155181 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-55f6479cff-fhj7l"] Jun 12 18:47:21.155687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155646 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="util" Jun 12 18:47:21.155687 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155663 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="util" Jun 12 18:47:21.155760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155698 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="extract" Jun 12 18:47:21.155760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155707 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="extract" Jun 12 18:47:21.155760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155724 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="pull" Jun 12 18:47:21.155760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155732 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="pull" Jun 12 18:47:21.155878 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.155803 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d801711e-3c78-4100-878a-f69159662671" containerName="extract" Jun 12 18:47:21.160337 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.160298 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.164943 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.164795 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 12 18:47:21.165267 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.165250 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mcgzg\"" Jun 12 18:47:21.182661 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.182638 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-55f6479cff-fhj7l"] Jun 12 18:47:21.249857 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.249826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpkcq\" (UniqueName: \"kubernetes.io/projected/0aa01331-abd9-4283-9c3c-a906d359a87a-kube-api-access-qpkcq\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.250005 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.249894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/0aa01331-abd9-4283-9c3c-a906d359a87a-redirect-config\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.350878 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.350851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpkcq\" (UniqueName: \"kubernetes.io/projected/0aa01331-abd9-4283-9c3c-a906d359a87a-kube-api-access-qpkcq\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.351015 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.350932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/0aa01331-abd9-4283-9c3c-a906d359a87a-redirect-config\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.351629 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.351612 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/0aa01331-abd9-4283-9c3c-a906d359a87a-redirect-config\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.371917 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.371894 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpkcq\" (UniqueName: \"kubernetes.io/projected/0aa01331-abd9-4283-9c3c-a906d359a87a-kube-api-access-qpkcq\") pod \"dashboard-redirect-55f6479cff-fhj7l\" (UID: \"0aa01331-abd9-4283-9c3c-a906d359a87a\") " pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.470933 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.470870 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" Jun 12 18:47:21.808295 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:21.808129 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-55f6479cff-fhj7l"] Jun 12 18:47:21.810496 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:47:21.810467 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa01331_abd9_4283_9c3c_a906d359a87a.slice/crio-1ab3b36e4c3950265f018259a34402c0dd2f87c63c4599f565872aafa1637390 WatchSource:0}: Error finding container 1ab3b36e4c3950265f018259a34402c0dd2f87c63c4599f565872aafa1637390: Status 404 returned error can't find the container with id 1ab3b36e4c3950265f018259a34402c0dd2f87c63c4599f565872aafa1637390 Jun 12 18:47:22.005089 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:22.005046 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" event={"ID":"0aa01331-abd9-4283-9c3c-a906d359a87a","Type":"ContainerStarted","Data":"1ab3b36e4c3950265f018259a34402c0dd2f87c63c4599f565872aafa1637390"} Jun 12 18:47:24.021992 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:24.021953 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" event={"ID":"0aa01331-abd9-4283-9c3c-a906d359a87a","Type":"ContainerStarted","Data":"b46ed871ead4820f44115fa8ba7a6d7ec65a1a97614edc56b5e52388abdadc7f"} Jun 12 18:47:24.041526 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:24.041435 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-55f6479cff-fhj7l" podStartSLOduration=1.07357252 podStartE2EDuration="3.041420846s" totalImagesPullingTime="1.967848326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:47:24.039144844 +0000 UTC m=+561.605422081" watchObservedRunningTime="2026-06-12 18:47:24.041420846 +0000 UTC m=+561.607698073" Jun 12 18:47:37.137621 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.137590 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q"] Jun 12 18:47:37.143516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.143482 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.147590 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.147563 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wlkvf\"" Jun 12 18:47:37.149051 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.149026 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 12 18:47:37.166443 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.166419 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q"] Jun 12 18:47:37.185296 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj4lz\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-kube-api-access-kj4lz\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185348 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185556 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185556 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185640 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185640 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.185640 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.185625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.286868 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.286868 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj4lz\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-kube-api-access-kj4lz\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.286982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287351 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287425 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287456 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287485 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.287758 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.287739 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.289369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.289349 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.289523 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.289506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.296615 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.296592 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.296899 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.296876 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj4lz\" (UniqueName: \"kubernetes.io/projected/049593e3-ea58-463a-9f0d-fd5c3557f8ba-kube-api-access-kj4lz\") pod \"data-science-gateway-data-science-gateway-class-74759f9795h475q\" (UID: \"049593e3-ea58-463a-9f0d-fd5c3557f8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.454463 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.454372 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:37.806131 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:37.806055 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q"] Jun 12 18:47:37.808698 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:47:37.808666 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049593e3_ea58_463a_9f0d_fd5c3557f8ba.slice/crio-cc0e9e772a9b0a7ad84522bbe2d6bf8f91d733c32059d333754d26b77dc613ef WatchSource:0}: Error finding container cc0e9e772a9b0a7ad84522bbe2d6bf8f91d733c32059d333754d26b77dc613ef: Status 404 returned error can't find the container with id cc0e9e772a9b0a7ad84522bbe2d6bf8f91d733c32059d333754d26b77dc613ef Jun 12 18:47:38.091553 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:38.091467 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" event={"ID":"049593e3-ea58-463a-9f0d-fd5c3557f8ba","Type":"ContainerStarted","Data":"cc0e9e772a9b0a7ad84522bbe2d6bf8f91d733c32059d333754d26b77dc613ef"} Jun 12 18:47:40.363736 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:40.363697 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:47:40.364006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:40.363774 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:47:40.364006 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:40.363802 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:47:41.109930 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:41.109890 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" event={"ID":"049593e3-ea58-463a-9f0d-fd5c3557f8ba","Type":"ContainerStarted","Data":"3416aed020be9b85ff29189c1b32b5f6c46fc52ed3a7b1ea736b9816357730b5"} Jun 12 18:47:41.136099 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:41.136051 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" podStartSLOduration=1.583452756 podStartE2EDuration="4.136035526s" totalImagesPullingTime="2.55258277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:47:41.13309469 +0000 UTC m=+578.699371948" watchObservedRunningTime="2026-06-12 18:47:41.136035526 +0000 UTC m=+578.702312761" Jun 12 18:47:41.455186 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:41.455144 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:41.460013 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:41.459988 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:42.114666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:42.114631 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:42.115615 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:42.115595 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795h475q" Jun 12 18:47:46.112350 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.112303 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:46.115705 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.115683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:46.119581 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.119558 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 12 18:47:46.119697 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.119558 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9xhnp\"" Jun 12 18:47:46.120641 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.120621 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 12 18:47:46.136070 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.136044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:46.266851 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.266818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrlj\" (UniqueName: \"kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj\") pod \"kuadrant-operator-catalog-m6gh6\" (UID: \"e4480b3c-79a1-421d-a8f8-95227ad85048\") " pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:46.367379 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.367273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrlj\" (UniqueName: \"kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj\") pod \"kuadrant-operator-catalog-m6gh6\" (UID: \"e4480b3c-79a1-421d-a8f8-95227ad85048\") " pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:46.378272 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.378245 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrlj\" (UniqueName: \"kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj\") pod \"kuadrant-operator-catalog-m6gh6\" (UID: \"e4480b3c-79a1-421d-a8f8-95227ad85048\") " pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:46.426336 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.426280 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:46.452643 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.452611 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:46.667177 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.667144 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2lrz"] Jun 12 18:47:46.671454 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.671435 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:46.679270 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.679242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2lrz"] Jun 12 18:47:46.767793 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.767770 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:46.769740 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:47:46.769715 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4480b3c_79a1_421d_a8f8_95227ad85048.slice/crio-e787905f7899c92fa8233c8c566381a5d548b2c1880fc28364f7576fd3c44680 WatchSource:0}: Error finding container e787905f7899c92fa8233c8c566381a5d548b2c1880fc28364f7576fd3c44680: Status 404 returned error can't find the container with id e787905f7899c92fa8233c8c566381a5d548b2c1880fc28364f7576fd3c44680 Jun 12 18:47:46.770779 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.770752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxlgf\" (UniqueName: \"kubernetes.io/projected/b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0-kube-api-access-zxlgf\") pod \"kuadrant-operator-catalog-h2lrz\" (UID: \"b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0\") " pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:46.871905 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.871880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxlgf\" (UniqueName: \"kubernetes.io/projected/b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0-kube-api-access-zxlgf\") pod \"kuadrant-operator-catalog-h2lrz\" (UID: \"b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0\") " pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:46.880748 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.880723 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxlgf\" (UniqueName: \"kubernetes.io/projected/b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0-kube-api-access-zxlgf\") pod \"kuadrant-operator-catalog-h2lrz\" (UID: \"b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0\") " pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:46.981174 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:46.981106 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:47.139953 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:47.139924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" event={"ID":"e4480b3c-79a1-421d-a8f8-95227ad85048","Type":"ContainerStarted","Data":"e787905f7899c92fa8233c8c566381a5d548b2c1880fc28364f7576fd3c44680"} Jun 12 18:47:47.318925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:47.318890 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2lrz"] Jun 12 18:47:47.320958 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:47:47.320930 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fcbbeb_48be_4153_b81d_77d2f8b8d1c0.slice/crio-06c7f11e421cdb598d825b523447db1452aa32c13b85dfa4da329d52584c026d WatchSource:0}: Error finding container 06c7f11e421cdb598d825b523447db1452aa32c13b85dfa4da329d52584c026d: Status 404 returned error can't find the container with id 06c7f11e421cdb598d825b523447db1452aa32c13b85dfa4da329d52584c026d Jun 12 18:47:48.147232 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:48.147193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" event={"ID":"b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0","Type":"ContainerStarted","Data":"06c7f11e421cdb598d825b523447db1452aa32c13b85dfa4da329d52584c026d"} Jun 12 18:47:50.159223 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.159190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" event={"ID":"e4480b3c-79a1-421d-a8f8-95227ad85048","Type":"ContainerStarted","Data":"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd"} Jun 12 18:47:50.159674 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.159247 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" podUID="e4480b3c-79a1-421d-a8f8-95227ad85048" containerName="registry-server" containerID="cri-o://4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd" gracePeriod=2 Jun 12 18:47:50.160923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.160898 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" event={"ID":"b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0","Type":"ContainerStarted","Data":"e2bdbf246c30615d818b963d98b679b7b700b071e542c152213f06e2e639559e"} Jun 12 18:47:50.177610 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.177569 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" podStartSLOduration=1.851162975 podStartE2EDuration="4.17755627s" totalImagesPullingTime="2.326393295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:47:50.175483379 +0000 UTC m=+587.741760615" watchObservedRunningTime="2026-06-12 18:47:50.17755627 +0000 UTC m=+587.743833523" Jun 12 18:47:50.191762 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.191721 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" podStartSLOduration=2.414292753 podStartE2EDuration="4.191711874s" totalImagesPullingTime="1.777419121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:47:50.19150455 +0000 UTC m=+587.757781785" watchObservedRunningTime="2026-06-12 18:47:50.191711874 +0000 UTC m=+587.757989152" Jun 12 18:47:50.501586 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.501562 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:50.612545 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.612513 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtrlj\" (UniqueName: \"kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj\") pod \"e4480b3c-79a1-421d-a8f8-95227ad85048\" (UID: \"e4480b3c-79a1-421d-a8f8-95227ad85048\") " Jun 12 18:47:50.614650 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.614628 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj" (OuterVolumeSpecName: "kube-api-access-rtrlj") pod "e4480b3c-79a1-421d-a8f8-95227ad85048" (UID: "e4480b3c-79a1-421d-a8f8-95227ad85048"). InnerVolumeSpecName "kube-api-access-rtrlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:47:50.714098 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:50.714069 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtrlj\" (UniqueName: \"kubernetes.io/projected/e4480b3c-79a1-421d-a8f8-95227ad85048-kube-api-access-rtrlj\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:47:51.166759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.166671 2577 generic.go:356] "Generic (PLEG): container finished" podID="e4480b3c-79a1-421d-a8f8-95227ad85048" containerID="4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd" exitCode=0 Jun 12 18:47:51.166759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.166724 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" Jun 12 18:47:51.167292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.166761 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" event={"ID":"e4480b3c-79a1-421d-a8f8-95227ad85048","Type":"ContainerDied","Data":"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd"} Jun 12 18:47:51.167292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.166844 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m6gh6" event={"ID":"e4480b3c-79a1-421d-a8f8-95227ad85048","Type":"ContainerDied","Data":"e787905f7899c92fa8233c8c566381a5d548b2c1880fc28364f7576fd3c44680"} Jun 12 18:47:51.167292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.166874 2577 scope.go:117] "RemoveContainer" containerID="4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd" Jun 12 18:47:51.175930 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.175914 2577 scope.go:117] "RemoveContainer" containerID="4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd" Jun 12 18:47:51.176167 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:47:51.176148 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd\": container with ID starting with 4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd not found: ID does not exist" containerID="4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd" Jun 12 18:47:51.176232 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.176174 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd"} err="failed to get container status \"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd\": rpc error: code = NotFound desc = could not find container \"4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd\": container with ID starting with 4ddd3cefc0c17f587154afabcc46013695cd6b7cf8e14e008b478560c237dbcd not found: ID does not exist" Jun 12 18:47:51.189239 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.189218 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:51.193751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:51.193730 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m6gh6"] Jun 12 18:47:52.900242 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:52.900193 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4480b3c-79a1-421d-a8f8-95227ad85048" path="/var/lib/kubelet/pods/e4480b3c-79a1-421d-a8f8-95227ad85048/volumes" Jun 12 18:47:56.981741 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:56.981709 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:56.982138 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:56.981775 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:57.003925 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:57.003899 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:47:57.219195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:47:57.219172 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h2lrz" Jun 12 18:48:01.105881 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.105849 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj"] Jun 12 18:48:01.106246 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.106207 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4480b3c-79a1-421d-a8f8-95227ad85048" containerName="registry-server" Jun 12 18:48:01.106246 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.106219 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4480b3c-79a1-421d-a8f8-95227ad85048" containerName="registry-server" Jun 12 18:48:01.106352 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.106297 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4480b3c-79a1-421d-a8f8-95227ad85048" containerName="registry-server" Jun 12 18:48:01.117890 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.117868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.123421 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.123391 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6qczc\"" Jun 12 18:48:01.124682 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.124660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj"] Jun 12 18:48:01.305872 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.305839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.306035 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.305878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77r2\" (UniqueName: \"kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.306035 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.305976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.406686 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.406657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.406824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.406703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.406824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.406726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77r2\" (UniqueName: \"kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.407046 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.407027 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.407088 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.407062 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.417438 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.417411 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77r2\" (UniqueName: \"kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.428168 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.428151 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:01.711337 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.711253 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl"] Jun 12 18:48:01.716204 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.716176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:01.727553 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.727529 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl"] Jun 12 18:48:01.780469 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.780446 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj"] Jun 12 18:48:01.781916 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:01.781891 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d50de42_2e58_4a9f_86ff_dda2c0dfa9f2.slice/crio-24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7 WatchSource:0}: Error finding container 24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7: Status 404 returned error can't find the container with id 24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7 Jun 12 18:48:01.911273 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.911247 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52bw8\" (UniqueName: \"kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:01.911392 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.911363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:01.911446 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:01.911414 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.012384 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.012294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52bw8\" (UniqueName: \"kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.012494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.012423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.012494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.012473 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.012772 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.012753 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.012846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.012789 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.023445 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.023419 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52bw8\" (UniqueName: \"kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.025158 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.025142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:02.222762 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.222730 2577 generic.go:356] "Generic (PLEG): container finished" podID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerID="ee594390201278fe2e15647e59738ffb37008bd4b9e2f15cca811fdac12c1610" exitCode=0 Jun 12 18:48:02.223107 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.222796 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" event={"ID":"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2","Type":"ContainerDied","Data":"ee594390201278fe2e15647e59738ffb37008bd4b9e2f15cca811fdac12c1610"} Jun 12 18:48:02.223107 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.222848 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" event={"ID":"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2","Type":"ContainerStarted","Data":"24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7"} Jun 12 18:48:02.314859 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.314785 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666"] Jun 12 18:48:02.319501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.319481 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.333896 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.333872 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666"] Jun 12 18:48:02.373899 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.373880 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl"] Jun 12 18:48:02.375753 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:02.375731 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f3b8b0_cbcd_400a_88ea_0a17211c93d7.slice/crio-2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139 WatchSource:0}: Error finding container 2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139: Status 404 returned error can't find the container with id 2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139 Jun 12 18:48:02.416397 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.416373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.416501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.416423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.416566 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.416510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjd7p\" (UniqueName: \"kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.517249 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.517221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjd7p\" (UniqueName: \"kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.517410 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.517389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.517489 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.517442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.517734 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.517713 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.517787 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.517748 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.534810 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.534785 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjd7p\" (UniqueName: \"kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.628329 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.628257 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:02.717449 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.717422 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42"] Jun 12 18:48:02.722861 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.722840 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.750973 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.750947 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42"] Jun 12 18:48:02.821451 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.821423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.821607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.821533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.821607 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.821595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77rbg\" (UniqueName: \"kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.922267 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.922241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.922386 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.922291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77rbg\" (UniqueName: \"kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.922386 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.922338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.922706 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.922660 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.922805 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.922768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.932826 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.932806 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77rbg\" (UniqueName: \"kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:02.991995 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:02.975726 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666"] Jun 12 18:48:02.995299 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:02.995279 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0457de_6a0b_4030_9568_9c98103d4353.slice/crio-97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a WatchSource:0}: Error finding container 97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a: Status 404 returned error can't find the container with id 97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a Jun 12 18:48:03.033208 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.033189 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:03.229428 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.229355 2577 generic.go:356] "Generic (PLEG): container finished" podID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerID="b5cb264cb9d2bfcfab4fc5bba58178716f2d6fd55fe41292a105a8f8f0d175af" exitCode=0 Jun 12 18:48:03.229820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.229449 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" event={"ID":"17f3b8b0-cbcd-400a-88ea-0a17211c93d7","Type":"ContainerDied","Data":"b5cb264cb9d2bfcfab4fc5bba58178716f2d6fd55fe41292a105a8f8f0d175af"} Jun 12 18:48:03.229820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.229507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" event={"ID":"17f3b8b0-cbcd-400a-88ea-0a17211c93d7","Type":"ContainerStarted","Data":"2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139"} Jun 12 18:48:03.231352 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.231327 2577 generic.go:356] "Generic (PLEG): container finished" podID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerID="97f75469c3d5fb820e6743b5a09d4fea11a44e204131aacf77af5f162ee5a36b" exitCode=0 Jun 12 18:48:03.231480 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.231415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" event={"ID":"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2","Type":"ContainerDied","Data":"97f75469c3d5fb820e6743b5a09d4fea11a44e204131aacf77af5f162ee5a36b"} Jun 12 18:48:03.232973 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.232954 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a0457de-6a0b-4030-9568-9c98103d4353" containerID="27022f1bd26005020820c3311f342b7df01d4f86a79b75ec1061e645e5959de8" exitCode=0 Jun 12 18:48:03.233056 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.232999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" event={"ID":"1a0457de-6a0b-4030-9568-9c98103d4353","Type":"ContainerDied","Data":"27022f1bd26005020820c3311f342b7df01d4f86a79b75ec1061e645e5959de8"} Jun 12 18:48:03.233056 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.233028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" event={"ID":"1a0457de-6a0b-4030-9568-9c98103d4353","Type":"ContainerStarted","Data":"97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a"} Jun 12 18:48:03.377102 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.377076 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42"] Jun 12 18:48:03.427188 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:03.427152 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda65fafe6_4fce_4cfd_a9b9_63da7d298fec.slice/crio-5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b WatchSource:0}: Error finding container 5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b: Status 404 returned error can't find the container with id 5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b Jun 12 18:48:03.451110 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.451080 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:48:03.451456 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:03.451434 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:48:04.240582 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.240437 2577 generic.go:356] "Generic (PLEG): container finished" podID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerID="204cc6c44b02cb3be53d707a1a44e9a6549dca4e58cac9beb2977c85172e0b1b" exitCode=0 Jun 12 18:48:04.240582 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.240504 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" event={"ID":"17f3b8b0-cbcd-400a-88ea-0a17211c93d7","Type":"ContainerDied","Data":"204cc6c44b02cb3be53d707a1a44e9a6549dca4e58cac9beb2977c85172e0b1b"} Jun 12 18:48:04.242325 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.242289 2577 generic.go:356] "Generic (PLEG): container finished" podID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerID="2cdaeca59deabdd3106bd6a21bfc2bee62b23d7dda214cc665cdc5530c2a5cab" exitCode=0 Jun 12 18:48:04.242413 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.242344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" event={"ID":"a65fafe6-4fce-4cfd-a9b9-63da7d298fec","Type":"ContainerDied","Data":"2cdaeca59deabdd3106bd6a21bfc2bee62b23d7dda214cc665cdc5530c2a5cab"} Jun 12 18:48:04.242413 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.242379 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" event={"ID":"a65fafe6-4fce-4cfd-a9b9-63da7d298fec","Type":"ContainerStarted","Data":"5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b"} Jun 12 18:48:04.244979 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.244960 2577 generic.go:356] "Generic (PLEG): container finished" podID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerID="a9c7e286004c70668fa26e88b8598438409738614727a2aee993ae3878e2bf37" exitCode=0 Jun 12 18:48:04.245069 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.245046 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" event={"ID":"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2","Type":"ContainerDied","Data":"a9c7e286004c70668fa26e88b8598438409738614727a2aee993ae3878e2bf37"} Jun 12 18:48:04.246841 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.246823 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a0457de-6a0b-4030-9568-9c98103d4353" containerID="c45d1bc30c0650e87d4df9cbc20c3315e8c4a772a5e3f54ca23c9b00d343518b" exitCode=0 Jun 12 18:48:04.246934 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:04.246883 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" event={"ID":"1a0457de-6a0b-4030-9568-9c98103d4353","Type":"ContainerDied","Data":"c45d1bc30c0650e87d4df9cbc20c3315e8c4a772a5e3f54ca23c9b00d343518b"} Jun 12 18:48:05.254159 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.254055 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a0457de-6a0b-4030-9568-9c98103d4353" containerID="4faf10219a092ab5e04c678cb278a363b214f099f02d173260776892d770c1c3" exitCode=0 Jun 12 18:48:05.254550 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.254147 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" event={"ID":"1a0457de-6a0b-4030-9568-9c98103d4353","Type":"ContainerDied","Data":"4faf10219a092ab5e04c678cb278a363b214f099f02d173260776892d770c1c3"} Jun 12 18:48:05.256203 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.256179 2577 generic.go:356] "Generic (PLEG): container finished" podID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerID="6b7d762e8f643e9cd3949a9dbba49dcc7cd2ffaa037f170ad2a0ef469c48a46a" exitCode=0 Jun 12 18:48:05.256327 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.256252 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" event={"ID":"17f3b8b0-cbcd-400a-88ea-0a17211c93d7","Type":"ContainerDied","Data":"6b7d762e8f643e9cd3949a9dbba49dcc7cd2ffaa037f170ad2a0ef469c48a46a"} Jun 12 18:48:05.257838 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.257820 2577 generic.go:356] "Generic (PLEG): container finished" podID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerID="cb262bbda604f4800050ea3007455476396c0ff0df6b87e1e47b1e46914dc4c3" exitCode=0 Jun 12 18:48:05.257936 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.257892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" event={"ID":"a65fafe6-4fce-4cfd-a9b9-63da7d298fec","Type":"ContainerDied","Data":"cb262bbda604f4800050ea3007455476396c0ff0df6b87e1e47b1e46914dc4c3"} Jun 12 18:48:05.508970 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.508918 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:05.545660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.545632 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle\") pod \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " Jun 12 18:48:05.545795 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.545684 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util\") pod \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " Jun 12 18:48:05.545795 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.545707 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d77r2\" (UniqueName: \"kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2\") pod \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\" (UID: \"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2\") " Jun 12 18:48:05.546051 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.546029 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle" (OuterVolumeSpecName: "bundle") pod "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" (UID: "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:05.547824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.547800 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2" (OuterVolumeSpecName: "kube-api-access-d77r2") pod "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" (UID: "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2"). InnerVolumeSpecName "kube-api-access-d77r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:05.552742 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.552712 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util" (OuterVolumeSpecName: "util") pod "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" (UID: "7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:05.646823 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.646800 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:05.646823 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.646822 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:05.646962 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:05.646832 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d77r2\" (UniqueName: \"kubernetes.io/projected/7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2-kube-api-access-d77r2\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.264278 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.264240 2577 generic.go:356] "Generic (PLEG): container finished" podID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerID="3a697b8a1f99a12c8959861f2eba94d19182e54fa994f51b400882ee10742f09" exitCode=0 Jun 12 18:48:06.264728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.264344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" event={"ID":"a65fafe6-4fce-4cfd-a9b9-63da7d298fec","Type":"ContainerDied","Data":"3a697b8a1f99a12c8959861f2eba94d19182e54fa994f51b400882ee10742f09"} Jun 12 18:48:06.266054 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.266035 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" Jun 12 18:48:06.266169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.266072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj" event={"ID":"7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2","Type":"ContainerDied","Data":"24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7"} Jun 12 18:48:06.266169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.266109 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24ee53b758f416679e8de5a7a3116b81671305c83e8cbf24f01b198bb5fc0bd7" Jun 12 18:48:06.511868 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.511843 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:06.560579 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.560500 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjd7p\" (UniqueName: \"kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p\") pod \"1a0457de-6a0b-4030-9568-9c98103d4353\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " Jun 12 18:48:06.560579 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.560560 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle\") pod \"1a0457de-6a0b-4030-9568-9c98103d4353\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " Jun 12 18:48:06.560794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.560618 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util\") pod \"1a0457de-6a0b-4030-9568-9c98103d4353\" (UID: \"1a0457de-6a0b-4030-9568-9c98103d4353\") " Jun 12 18:48:06.561013 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.560979 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle" (OuterVolumeSpecName: "bundle") pod "1a0457de-6a0b-4030-9568-9c98103d4353" (UID: "1a0457de-6a0b-4030-9568-9c98103d4353"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:06.562532 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.562510 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p" (OuterVolumeSpecName: "kube-api-access-pjd7p") pod "1a0457de-6a0b-4030-9568-9c98103d4353" (UID: "1a0457de-6a0b-4030-9568-9c98103d4353"). InnerVolumeSpecName "kube-api-access-pjd7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:06.565957 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.565933 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util" (OuterVolumeSpecName: "util") pod "1a0457de-6a0b-4030-9568-9c98103d4353" (UID: "1a0457de-6a0b-4030-9568-9c98103d4353"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:06.601070 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.601052 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:06.661243 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661214 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util\") pod \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " Jun 12 18:48:06.661378 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661266 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle\") pod \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " Jun 12 18:48:06.661378 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661361 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52bw8\" (UniqueName: \"kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8\") pod \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\" (UID: \"17f3b8b0-cbcd-400a-88ea-0a17211c93d7\") " Jun 12 18:48:06.661612 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661595 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjd7p\" (UniqueName: \"kubernetes.io/projected/1a0457de-6a0b-4030-9568-9c98103d4353-kube-api-access-pjd7p\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.661650 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661617 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.661650 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661632 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a0457de-6a0b-4030-9568-9c98103d4353-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.661819 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.661791 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle" (OuterVolumeSpecName: "bundle") pod "17f3b8b0-cbcd-400a-88ea-0a17211c93d7" (UID: "17f3b8b0-cbcd-400a-88ea-0a17211c93d7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:06.663266 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.663244 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8" (OuterVolumeSpecName: "kube-api-access-52bw8") pod "17f3b8b0-cbcd-400a-88ea-0a17211c93d7" (UID: "17f3b8b0-cbcd-400a-88ea-0a17211c93d7"). InnerVolumeSpecName "kube-api-access-52bw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:06.666458 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.666439 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util" (OuterVolumeSpecName: "util") pod "17f3b8b0-cbcd-400a-88ea-0a17211c93d7" (UID: "17f3b8b0-cbcd-400a-88ea-0a17211c93d7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:06.762298 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.762274 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.762298 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.762296 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:06.762450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:06.762305 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52bw8\" (UniqueName: \"kubernetes.io/projected/17f3b8b0-cbcd-400a-88ea-0a17211c93d7-kube-api-access-52bw8\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:07.273060 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.272978 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" Jun 12 18:48:07.273060 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.273005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl" event={"ID":"17f3b8b0-cbcd-400a-88ea-0a17211c93d7","Type":"ContainerDied","Data":"2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139"} Jun 12 18:48:07.273060 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.273055 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cb9012612129f7ba0efea65ba93d5b8daffbdc191634e06eaf1442ebfe14139" Jun 12 18:48:07.275099 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.275072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" event={"ID":"1a0457de-6a0b-4030-9568-9c98103d4353","Type":"ContainerDied","Data":"97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a"} Jun 12 18:48:07.275099 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.275094 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666" Jun 12 18:48:07.275385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.275119 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97ff5de92781c0bf68ffdbfb188f069030bec9065e34f993f0c286db108bce0a" Jun 12 18:48:07.486261 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.486238 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:07.570571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.570488 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle\") pod \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " Jun 12 18:48:07.570571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.570538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77rbg\" (UniqueName: \"kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg\") pod \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " Jun 12 18:48:07.570571 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.570561 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util\") pod \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\" (UID: \"a65fafe6-4fce-4cfd-a9b9-63da7d298fec\") " Jun 12 18:48:07.571017 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.570990 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle" (OuterVolumeSpecName: "bundle") pod "a65fafe6-4fce-4cfd-a9b9-63da7d298fec" (UID: "a65fafe6-4fce-4cfd-a9b9-63da7d298fec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:07.572625 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.572601 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg" (OuterVolumeSpecName: "kube-api-access-77rbg") pod "a65fafe6-4fce-4cfd-a9b9-63da7d298fec" (UID: "a65fafe6-4fce-4cfd-a9b9-63da7d298fec"). InnerVolumeSpecName "kube-api-access-77rbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:07.578809 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.578782 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util" (OuterVolumeSpecName: "util") pod "a65fafe6-4fce-4cfd-a9b9-63da7d298fec" (UID: "a65fafe6-4fce-4cfd-a9b9-63da7d298fec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:07.671289 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.671241 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77rbg\" (UniqueName: \"kubernetes.io/projected/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-kube-api-access-77rbg\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:07.671289 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.671281 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:07.671289 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:07.671290 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65fafe6-4fce-4cfd-a9b9-63da7d298fec-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:08.281762 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:08.281733 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" Jun 12 18:48:08.282144 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:08.281739 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42" event={"ID":"a65fafe6-4fce-4cfd-a9b9-63da7d298fec","Type":"ContainerDied","Data":"5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b"} Jun 12 18:48:08.282144 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:08.281857 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5cf4f981bca5ef83fc2bfbaedf205a21c6b8c93effac92cf934f761adc19757b" Jun 12 18:48:13.986459 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986419 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tndr"] Jun 12 18:48:13.986915 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986900 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="util" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986919 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="util" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986930 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="extract" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986941 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="extract" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986953 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="extract" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986963 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="extract" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986980 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="extract" Jun 12 18:48:13.986989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.986991 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987001 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987009 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987022 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987030 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987043 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987052 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987066 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987075 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987083 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987091 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987100 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987108 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987118 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987125 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="pull" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987141 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987182 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="util" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987270 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987284 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="17f3b8b0-cbcd-400a-88ea-0a17211c93d7" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987295 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a65fafe6-4fce-4cfd-a9b9-63da7d298fec" containerName="extract" Jun 12 18:48:13.987369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.987304 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0457de-6a0b-4030-9568-9c98103d4353" containerName="extract" Jun 12 18:48:13.993519 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.993499 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:13.996200 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:13.996181 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tbmqd\"" Jun 12 18:48:14.015596 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.015571 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tndr"] Jun 12 18:48:14.028263 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.028235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwdv4\" (UniqueName: \"kubernetes.io/projected/222b1e9d-e3b7-41e7-aa55-07305ea343e8-kube-api-access-jwdv4\") pod \"authorino-operator-75cc8d5cb7-5tndr\" (UID: \"222b1e9d-e3b7-41e7-aa55-07305ea343e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:14.129437 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.129409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwdv4\" (UniqueName: \"kubernetes.io/projected/222b1e9d-e3b7-41e7-aa55-07305ea343e8-kube-api-access-jwdv4\") pod \"authorino-operator-75cc8d5cb7-5tndr\" (UID: \"222b1e9d-e3b7-41e7-aa55-07305ea343e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:14.149684 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.149660 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwdv4\" (UniqueName: \"kubernetes.io/projected/222b1e9d-e3b7-41e7-aa55-07305ea343e8-kube-api-access-jwdv4\") pod \"authorino-operator-75cc8d5cb7-5tndr\" (UID: \"222b1e9d-e3b7-41e7-aa55-07305ea343e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:14.303760 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.303690 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:14.663365 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.663336 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tndr"] Jun 12 18:48:14.663970 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:14.663945 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222b1e9d_e3b7_41e7_aa55_07305ea343e8.slice/crio-ab173a243dc7886d0fca2b63051c008a55beeb4bcdc16e1bb06972463ec96ff0 WatchSource:0}: Error finding container ab173a243dc7886d0fca2b63051c008a55beeb4bcdc16e1bb06972463ec96ff0: Status 404 returned error can't find the container with id ab173a243dc7886d0fca2b63051c008a55beeb4bcdc16e1bb06972463ec96ff0 Jun 12 18:48:14.666465 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:14.666448 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 18:48:15.321394 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.321353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" event={"ID":"222b1e9d-e3b7-41e7-aa55-07305ea343e8","Type":"ContainerStarted","Data":"ab173a243dc7886d0fca2b63051c008a55beeb4bcdc16e1bb06972463ec96ff0"} Jun 12 18:48:15.805426 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.805385 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-784bdf947d-86wfv"] Jun 12 18:48:15.809275 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.809250 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.828551 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.828526 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784bdf947d-86wfv"] Jun 12 18:48:15.842876 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.842851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-service-ca\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.842996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.842893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.842996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.842916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-oauth-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.842996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.842978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94tdf\" (UniqueName: \"kubernetes.io/projected/5b593e2c-39e2-48ae-8d57-25c803fc79aa-kube-api-access-94tdf\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.843137 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.843026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.843137 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.843053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-trusted-ca-bundle\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.843137 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.843083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-oauth-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.943739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-service-ca\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.943739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.944195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-oauth-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.944195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94tdf\" (UniqueName: \"kubernetes.io/projected/5b593e2c-39e2-48ae-8d57-25c803fc79aa-kube-api-access-94tdf\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.944195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.944195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.943997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-trusted-ca-bundle\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.944195 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.944024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-oauth-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.945182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.944565 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-service-ca\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.945182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.944934 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-oauth-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.945182 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.944995 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.945723 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.945693 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b593e2c-39e2-48ae-8d57-25c803fc79aa-trusted-ca-bundle\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.946989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.946952 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-serving-cert\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.947412 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.947390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b593e2c-39e2-48ae-8d57-25c803fc79aa-console-oauth-config\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:15.954827 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:15.954805 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94tdf\" (UniqueName: \"kubernetes.io/projected/5b593e2c-39e2-48ae-8d57-25c803fc79aa-kube-api-access-94tdf\") pod \"console-784bdf947d-86wfv\" (UID: \"5b593e2c-39e2-48ae-8d57-25c803fc79aa\") " pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:16.120928 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:16.120826 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:17.281713 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:17.281681 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b593e2c_39e2_48ae_8d57_25c803fc79aa.slice/crio-642fe4e54f35bc59d6620ec04ed7bd44bba4944c2c75d7ff51608ddf940cae5a WatchSource:0}: Error finding container 642fe4e54f35bc59d6620ec04ed7bd44bba4944c2c75d7ff51608ddf940cae5a: Status 404 returned error can't find the container with id 642fe4e54f35bc59d6620ec04ed7bd44bba4944c2c75d7ff51608ddf940cae5a Jun 12 18:48:17.282240 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:17.282218 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784bdf947d-86wfv"] Jun 12 18:48:17.334863 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:17.334835 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" event={"ID":"222b1e9d-e3b7-41e7-aa55-07305ea343e8","Type":"ContainerStarted","Data":"acbc2058a6e49bdfa4e485cf9cd4700a96b5be4d9ef8609d03496cc9f2c55a3c"} Jun 12 18:48:17.335081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:17.334926 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:17.336247 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:17.336218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784bdf947d-86wfv" event={"ID":"5b593e2c-39e2-48ae-8d57-25c803fc79aa","Type":"ContainerStarted","Data":"642fe4e54f35bc59d6620ec04ed7bd44bba4944c2c75d7ff51608ddf940cae5a"} Jun 12 18:48:17.384215 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:17.384170 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" podStartSLOduration=2.094460449 podStartE2EDuration="4.384153502s" totalImagesPullingTime="2.289693053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:48:17.381909328 +0000 UTC m=+614.948186576" watchObservedRunningTime="2026-06-12 18:48:17.384153502 +0000 UTC m=+614.950430736" Jun 12 18:48:18.343966 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:18.343933 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784bdf947d-86wfv" event={"ID":"5b593e2c-39e2-48ae-8d57-25c803fc79aa","Type":"ContainerStarted","Data":"6c9da57a5532395f64f55c097457e9b78026266bca1b26fd6dd4be099ea599b1"} Jun 12 18:48:18.369719 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:18.369669 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-784bdf947d-86wfv" podStartSLOduration=3.369654565 podStartE2EDuration="3.369654565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:48:18.367338161 +0000 UTC m=+615.933615393" watchObservedRunningTime="2026-06-12 18:48:18.369654565 +0000 UTC m=+615.935931799" Jun 12 18:48:26.121597 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:26.121517 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:26.121597 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:26.121591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:26.126252 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:26.126230 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:26.389464 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:26.389437 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784bdf947d-86wfv" Jun 12 18:48:26.440004 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:26.439970 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:48:28.346406 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:28.346373 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tndr" Jun 12 18:48:36.020228 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.020194 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k"] Jun 12 18:48:36.023811 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.023788 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.026732 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.026711 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8dlhh\"" Jun 12 18:48:36.040485 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.040462 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k"] Jun 12 18:48:36.118075 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.118048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.118187 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.118102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hctkn\" (UniqueName: \"kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.143351 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.143327 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k"] Jun 12 18:48:36.143559 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:48:36.143541 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-hctkn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" podUID="0b85e421-37f8-41a1-9d23-e9b709296481" Jun 12 18:48:36.219131 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.219100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hctkn\" (UniqueName: \"kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.219248 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.219188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.219579 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.219561 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.237559 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.237530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hctkn\" (UniqueName: \"kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j2d4k\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.443338 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.443298 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.451579 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.451555 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:36.522211 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.522184 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hctkn\" (UniqueName: \"kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn\") pod \"0b85e421-37f8-41a1-9d23-e9b709296481\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " Jun 12 18:48:36.522367 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.522220 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume\") pod \"0b85e421-37f8-41a1-9d23-e9b709296481\" (UID: \"0b85e421-37f8-41a1-9d23-e9b709296481\") " Jun 12 18:48:36.522549 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.522526 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b85e421-37f8-41a1-9d23-e9b709296481" (UID: "0b85e421-37f8-41a1-9d23-e9b709296481"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:36.524170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.524144 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn" (OuterVolumeSpecName: "kube-api-access-hctkn") pod "0b85e421-37f8-41a1-9d23-e9b709296481" (UID: "0b85e421-37f8-41a1-9d23-e9b709296481"). InnerVolumeSpecName "kube-api-access-hctkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:36.623813 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.623787 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hctkn\" (UniqueName: \"kubernetes.io/projected/0b85e421-37f8-41a1-9d23-e9b709296481-kube-api-access-hctkn\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:36.623813 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.623811 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b85e421-37f8-41a1-9d23-e9b709296481-extensions-socket-volume\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:36.640824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.640797 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k"] Jun 12 18:48:36.654081 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.654050 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k"] Jun 12 18:48:36.719739 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.719666 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:36.723083 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.723059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.745093 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.745068 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:36.825685 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.825644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.825862 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.825717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfbd\" (UniqueName: \"kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.899928 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.899877 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b85e421-37f8-41a1-9d23-e9b709296481" path="/var/lib/kubelet/pods/0b85e421-37f8-41a1-9d23-e9b709296481/volumes" Jun 12 18:48:36.926420 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.926396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.926542 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.926442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfbd\" (UniqueName: \"kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.926772 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.926754 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:36.937209 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:36.937181 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfbd\" (UniqueName: \"kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vzb5v\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:37.033694 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:37.033628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:37.381024 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:37.380996 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:37.382576 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:48:37.382545 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f8f782_125a_4db2_a920_7ac26c43c260.slice/crio-bcf47d9e8dbcfef3684fb6a0d6b5cf129ed17e93135ae5fa702daf0587dc9070 WatchSource:0}: Error finding container bcf47d9e8dbcfef3684fb6a0d6b5cf129ed17e93135ae5fa702daf0587dc9070: Status 404 returned error can't find the container with id bcf47d9e8dbcfef3684fb6a0d6b5cf129ed17e93135ae5fa702daf0587dc9070 Jun 12 18:48:37.450049 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:37.450020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" event={"ID":"42f8f782-125a-4db2-a920-7ac26c43c260","Type":"ContainerStarted","Data":"bcf47d9e8dbcfef3684fb6a0d6b5cf129ed17e93135ae5fa702daf0587dc9070"} Jun 12 18:48:37.450153 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:37.450059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j2d4k" Jun 12 18:48:42.481726 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:42.481682 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" event={"ID":"42f8f782-125a-4db2-a920-7ac26c43c260","Type":"ContainerStarted","Data":"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5"} Jun 12 18:48:42.482117 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:42.481786 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:42.505094 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:42.505037 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" podStartSLOduration=2.327801892 podStartE2EDuration="6.505020838s" totalImagesPullingTime="4.177218946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:48:42.504835508 +0000 UTC m=+640.071112746" watchObservedRunningTime="2026-06-12 18:48:42.505020838 +0000 UTC m=+640.071298111" Jun 12 18:48:48.922933 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:48.922900 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:48.923345 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:48.923097 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" podUID="42f8f782-125a-4db2-a920-7ac26c43c260" containerName="manager" containerID="cri-o://73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5" gracePeriod=10 Jun 12 18:48:48.924891 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:48.924864 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:49.308065 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.308039 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:49.339360 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.339328 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume\") pod \"42f8f782-125a-4db2-a920-7ac26c43c260\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " Jun 12 18:48:49.339471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.339392 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwfbd\" (UniqueName: \"kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd\") pod \"42f8f782-125a-4db2-a920-7ac26c43c260\" (UID: \"42f8f782-125a-4db2-a920-7ac26c43c260\") " Jun 12 18:48:49.339903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.339872 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "42f8f782-125a-4db2-a920-7ac26c43c260" (UID: "42f8f782-125a-4db2-a920-7ac26c43c260"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:48:49.341725 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.341690 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd" (OuterVolumeSpecName: "kube-api-access-cwfbd") pod "42f8f782-125a-4db2-a920-7ac26c43c260" (UID: "42f8f782-125a-4db2-a920-7ac26c43c260"). InnerVolumeSpecName "kube-api-access-cwfbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:49.440976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.440952 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwfbd\" (UniqueName: \"kubernetes.io/projected/42f8f782-125a-4db2-a920-7ac26c43c260-kube-api-access-cwfbd\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:49.440976 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.440974 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f8f782-125a-4db2-a920-7ac26c43c260-extensions-socket-volume\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:49.519575 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.519549 2577 generic.go:356] "Generic (PLEG): container finished" podID="42f8f782-125a-4db2-a920-7ac26c43c260" containerID="73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5" exitCode=0 Jun 12 18:48:49.519715 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.519589 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" Jun 12 18:48:49.519715 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.519631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" event={"ID":"42f8f782-125a-4db2-a920-7ac26c43c260","Type":"ContainerDied","Data":"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5"} Jun 12 18:48:49.519715 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.519681 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v" event={"ID":"42f8f782-125a-4db2-a920-7ac26c43c260","Type":"ContainerDied","Data":"bcf47d9e8dbcfef3684fb6a0d6b5cf129ed17e93135ae5fa702daf0587dc9070"} Jun 12 18:48:49.519715 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.519703 2577 scope.go:117] "RemoveContainer" containerID="73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5" Jun 12 18:48:49.529517 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.529501 2577 scope.go:117] "RemoveContainer" containerID="73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5" Jun 12 18:48:49.529762 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:48:49.529745 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5\": container with ID starting with 73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5 not found: ID does not exist" containerID="73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5" Jun 12 18:48:49.529855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.529799 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5"} err="failed to get container status \"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5\": rpc error: code = NotFound desc = could not find container \"73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5\": container with ID starting with 73212304fbba0ab599960ad230840b022ef38e7c1fa0a51a256160a9fcd548e5 not found: ID does not exist" Jun 12 18:48:49.549172 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.549118 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:49.551341 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:49.551306 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vzb5v"] Jun 12 18:48:50.899543 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:50.899511 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f8f782-125a-4db2-a920-7ac26c43c260" path="/var/lib/kubelet/pods/42f8f782-125a-4db2-a920-7ac26c43c260/volumes" Jun 12 18:48:51.464722 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.464683 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b55d6fbd-kh6w5" podUID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" containerName="console" containerID="cri-o://9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105" gracePeriod=15 Jun 12 18:48:51.833906 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.833883 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:48:51.858295 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858402 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858345 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858461 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858442 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858502 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858473 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858540 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858519 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858578 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858539 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858578 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858570 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf76s\" (UniqueName: \"kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s\") pod \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\" (UID: \"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3\") " Jun 12 18:48:51.858681 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858652 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:48:51.859051 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858880 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-oauth-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.859051 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.858895 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config" (OuterVolumeSpecName: "console-config") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:48:51.859249 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.859051 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca" (OuterVolumeSpecName: "service-ca") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:48:51.859249 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.859101 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:48:51.860588 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.860568 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:48:51.860685 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.860602 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s" (OuterVolumeSpecName: "kube-api-access-wf76s") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "kube-api-access-wf76s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:48:51.860728 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.860682 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" (UID: "47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:48:51.959730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959706 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.959730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959729 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-service-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.960082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959741 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf76s\" (UniqueName: \"kubernetes.io/projected/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-kube-api-access-wf76s\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.960082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959752 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-serving-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.960082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959761 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-console-oauth-config\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:51.960082 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:51.959769 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3-trusted-ca-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:48:52.538444 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.538413 2577 generic.go:356] "Generic (PLEG): container finished" podID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" containerID="9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105" exitCode=0 Jun 12 18:48:52.538619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.538463 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55d6fbd-kh6w5" Jun 12 18:48:52.538619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.538498 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55d6fbd-kh6w5" event={"ID":"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3","Type":"ContainerDied","Data":"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105"} Jun 12 18:48:52.538619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.538550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55d6fbd-kh6w5" event={"ID":"47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3","Type":"ContainerDied","Data":"05279149f0a7103239edd117a37d371ceb6ddf45bfd8dea55ddad3cc954b9ba2"} Jun 12 18:48:52.538619 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.538577 2577 scope.go:117] "RemoveContainer" containerID="9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105" Jun 12 18:48:52.549198 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.549168 2577 scope.go:117] "RemoveContainer" containerID="9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105" Jun 12 18:48:52.549603 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:48:52.549537 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105\": container with ID starting with 9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105 not found: ID does not exist" containerID="9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105" Jun 12 18:48:52.549603 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.549572 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105"} err="failed to get container status \"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105\": rpc error: code = NotFound desc = could not find container \"9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105\": container with ID starting with 9627368cf61f1ad931d1967add0236190c663ed96f049c2775e2966664651105 not found: ID does not exist" Jun 12 18:48:52.566025 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.565974 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:48:52.577286 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.577264 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b55d6fbd-kh6w5"] Jun 12 18:48:52.900819 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:48:52.900787 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" path="/var/lib/kubelet/pods/47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3/volumes" Jun 12 18:49:05.241558 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241526 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2"] Jun 12 18:49:05.241947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241862 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" containerName="console" Jun 12 18:49:05.241947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241874 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" containerName="console" Jun 12 18:49:05.241947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241905 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f8f782-125a-4db2-a920-7ac26c43c260" containerName="manager" Jun 12 18:49:05.241947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241912 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f8f782-125a-4db2-a920-7ac26c43c260" containerName="manager" Jun 12 18:49:05.242093 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241971 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f8f782-125a-4db2-a920-7ac26c43c260" containerName="manager" Jun 12 18:49:05.242093 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.241981 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e5a5b0-e5a2-43c7-97fb-9d11b376a6c3" containerName="console" Jun 12 18:49:05.246443 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.246424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.251307 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.251284 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sfpbj\"" Jun 12 18:49:05.257747 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.257724 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2"] Jun 12 18:49:05.365881 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.365845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.365881 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.365879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.365913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwz9x\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-kube-api-access-qwz9x\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.365955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.365989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.366007 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.366038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.366055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.366095 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.366085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467614 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467614 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467614 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467614 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467704 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwz9x\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-kube-api-access-qwz9x\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467787 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.467846 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.467841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.468074 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.468053 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.468124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.468072 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.468405 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.468386 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.469751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.469724 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.469903 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.469859 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.476522 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.476502 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.476759 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.476741 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwz9x\" (UniqueName: \"kubernetes.io/projected/739eb282-2e8b-4a8a-9164-52ba1fb0a2a7-kube-api-access-qwz9x\") pod \"maas-default-gateway-openshift-default-687ff6996-69bs2\" (UID: \"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.557916 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.557842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:05.902584 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.902556 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2"] Jun 12 18:49:05.907647 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:49:05.907415 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739eb282_2e8b_4a8a_9164_52ba1fb0a2a7.slice/crio-806b2be211302e7a01b2114c4437532da8eed3cf74e5aee86f05d3243c3fa660 WatchSource:0}: Error finding container 806b2be211302e7a01b2114c4437532da8eed3cf74e5aee86f05d3243c3fa660: Status 404 returned error can't find the container with id 806b2be211302e7a01b2114c4437532da8eed3cf74e5aee86f05d3243c3fa660 Jun 12 18:49:05.910642 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.910609 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:49:05.910717 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.910679 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:49:05.910717 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:05.910710 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:49:06.617641 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:06.617606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" event={"ID":"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7","Type":"ContainerStarted","Data":"ff9539ff1d2b62b2eeda9d53db5f41b2018da09e00d149f2d9a47ed422227178"} Jun 12 18:49:06.617999 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:06.617657 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" event={"ID":"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7","Type":"ContainerStarted","Data":"806b2be211302e7a01b2114c4437532da8eed3cf74e5aee86f05d3243c3fa660"} Jun 12 18:49:06.646291 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:06.646243 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podStartSLOduration=1.646229239 podStartE2EDuration="1.646229239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:49:06.642462227 +0000 UTC m=+664.208739461" watchObservedRunningTime="2026-06-12 18:49:06.646229239 +0000 UTC m=+664.212506474" Jun 12 18:49:07.558569 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:07.558535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:07.563373 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:07.563350 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:07.623527 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:07.623502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:07.624245 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:07.624230 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:49:18.213234 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.213192 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:49:18.216471 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.216451 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.221262 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.221234 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6qczc\"" Jun 12 18:49:18.221412 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.221299 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 12 18:49:18.230435 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.230404 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:49:18.258519 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.258490 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:49:18.280589 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.280562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tq8w\" (UniqueName: \"kubernetes.io/projected/ff21b7d8-4915-4af4-9946-1b0bf4781155-kube-api-access-5tq8w\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.280719 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.280598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ff21b7d8-4915-4af4-9946-1b0bf4781155-config-file\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.381252 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.381224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tq8w\" (UniqueName: \"kubernetes.io/projected/ff21b7d8-4915-4af4-9946-1b0bf4781155-kube-api-access-5tq8w\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.381432 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.381262 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ff21b7d8-4915-4af4-9946-1b0bf4781155-config-file\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.381897 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.381877 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ff21b7d8-4915-4af4-9946-1b0bf4781155-config-file\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.390511 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.390482 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tq8w\" (UniqueName: \"kubernetes.io/projected/ff21b7d8-4915-4af4-9946-1b0bf4781155-kube-api-access-5tq8w\") pod \"limitador-limitador-54d6d988d5-26tc6\" (UID: \"ff21b7d8-4915-4af4-9946-1b0bf4781155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.527962 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.527887 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:18.830707 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.830613 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:49:18.833548 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.833533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:49:18.836576 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.836555 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xshx8\"" Jun 12 18:49:18.841923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.841902 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:49:18.870678 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.870655 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:49:18.872106 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:49:18.872080 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff21b7d8_4915_4af4_9946_1b0bf4781155.slice/crio-c9f59271ebed20ef0cde8e244c0744a77c9444b0aeb1ca1809dee187ae0d0df8 WatchSource:0}: Error finding container c9f59271ebed20ef0cde8e244c0744a77c9444b0aeb1ca1809dee187ae0d0df8: Status 404 returned error can't find the container with id c9f59271ebed20ef0cde8e244c0744a77c9444b0aeb1ca1809dee187ae0d0df8 Jun 12 18:49:18.987747 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:18.987714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2bn\" (UniqueName: \"kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn\") pod \"authorino-75d556bd77-qbz9w\" (UID: \"95a87ec4-ba31-4198-b579-f273f0d0b18c\") " pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:49:19.088707 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.088619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2bn\" (UniqueName: \"kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn\") pod \"authorino-75d556bd77-qbz9w\" (UID: \"95a87ec4-ba31-4198-b579-f273f0d0b18c\") " pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:49:19.097407 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.097382 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2bn\" (UniqueName: \"kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn\") pod \"authorino-75d556bd77-qbz9w\" (UID: \"95a87ec4-ba31-4198-b579-f273f0d0b18c\") " pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:49:19.144301 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.144272 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:49:19.505501 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.505463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:49:19.511680 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:49:19.511646 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95a87ec4_ba31_4198_b579_f273f0d0b18c.slice/crio-c5761f61cd59a592e7824317ce675b3ff2ec79f515b5152d74c29770836023b3 WatchSource:0}: Error finding container c5761f61cd59a592e7824317ce675b3ff2ec79f515b5152d74c29770836023b3: Status 404 returned error can't find the container with id c5761f61cd59a592e7824317ce675b3ff2ec79f515b5152d74c29770836023b3 Jun 12 18:49:19.689944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.689908 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbz9w" event={"ID":"95a87ec4-ba31-4198-b579-f273f0d0b18c","Type":"ContainerStarted","Data":"c5761f61cd59a592e7824317ce675b3ff2ec79f515b5152d74c29770836023b3"} Jun 12 18:49:19.691262 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:19.691235 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" event={"ID":"ff21b7d8-4915-4af4-9946-1b0bf4781155","Type":"ContainerStarted","Data":"c9f59271ebed20ef0cde8e244c0744a77c9444b0aeb1ca1809dee187ae0d0df8"} Jun 12 18:49:23.720060 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:23.720021 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbz9w" event={"ID":"95a87ec4-ba31-4198-b579-f273f0d0b18c","Type":"ContainerStarted","Data":"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9"} Jun 12 18:49:23.721550 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:23.721515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" event={"ID":"ff21b7d8-4915-4af4-9946-1b0bf4781155","Type":"ContainerStarted","Data":"cb2c2d2a91c96939f3ce9ce2128493376e0ab3328aa4acc425486bd3e34ee8d5"} Jun 12 18:49:23.721649 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:23.721570 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:23.743110 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:23.743064 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qbz9w" podStartSLOduration=2.130342934 podStartE2EDuration="5.743050496s" totalImagesPullingTime="3.612707562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:49:23.741828892 +0000 UTC m=+681.308106127" watchObservedRunningTime="2026-06-12 18:49:23.743050496 +0000 UTC m=+681.309327724" Jun 12 18:49:23.765850 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:23.765807 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" podStartSLOduration=1.5146478970000001 podStartE2EDuration="5.76579341s" totalImagesPullingTime="4.251145513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:49:23.762140634 +0000 UTC m=+681.328417869" watchObservedRunningTime="2026-06-12 18:49:23.76579341 +0000 UTC m=+681.332070645" Jun 12 18:49:34.727931 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:34.727896 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-26tc6" Jun 12 18:49:50.817494 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.817457 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79"] Jun 12 18:49:50.823214 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.823192 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.826019 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.825996 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 12 18:49:50.827277 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.827253 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 12 18:49:50.827477 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.827256 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7llmc\"" Jun 12 18:49:50.831413 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.831390 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79"] Jun 12 18:49:50.860889 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.860853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.861068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.860903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmm9d\" (UniqueName: \"kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.861068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.860967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.962197 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.962156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.962399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.962208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.962399 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.962240 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmm9d\" (UniqueName: \"kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.962639 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.962617 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.962688 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.962624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:50.971336 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:50.971289 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmm9d\" (UniqueName: \"kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:51.133907 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:51.133876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:51.498265 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:51.498236 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79"] Jun 12 18:49:51.499535 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:49:51.499499 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654aec53_a9aa_4b87_95d3_0881ccf04fa5.slice/crio-14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589 WatchSource:0}: Error finding container 14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589: Status 404 returned error can't find the container with id 14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589 Jun 12 18:49:51.882040 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:51.882004 2577 generic.go:356] "Generic (PLEG): container finished" podID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerID="8111cf49851cd866aeb4a94f0143c909fe85d930e3142499fc420cea91d14149" exitCode=0 Jun 12 18:49:51.882426 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:51.882078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" event={"ID":"654aec53-a9aa-4b87-95d3-0881ccf04fa5","Type":"ContainerDied","Data":"8111cf49851cd866aeb4a94f0143c909fe85d930e3142499fc420cea91d14149"} Jun 12 18:49:51.882426 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:51.882133 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" event={"ID":"654aec53-a9aa-4b87-95d3-0881ccf04fa5","Type":"ContainerStarted","Data":"14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589"} Jun 12 18:49:52.889386 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:52.889305 2577 generic.go:356] "Generic (PLEG): container finished" podID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerID="7b2af70983b7667d1a8914b830c0512ff67ac63c1c479fd7eae54bfccf5c3820" exitCode=0 Jun 12 18:49:52.889386 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:52.889362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" event={"ID":"654aec53-a9aa-4b87-95d3-0881ccf04fa5","Type":"ContainerDied","Data":"7b2af70983b7667d1a8914b830c0512ff67ac63c1c479fd7eae54bfccf5c3820"} Jun 12 18:49:53.897617 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:53.897583 2577 generic.go:356] "Generic (PLEG): container finished" podID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerID="3cbf20ab1172e4e9b857789d87800dd3736a78c0bfe89c21e6d1ea1ca655b5b4" exitCode=0 Jun 12 18:49:53.897989 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:53.897665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" event={"ID":"654aec53-a9aa-4b87-95d3-0881ccf04fa5","Type":"ContainerDied","Data":"3cbf20ab1172e4e9b857789d87800dd3736a78c0bfe89c21e6d1ea1ca655b5b4"} Jun 12 18:49:55.136351 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.136302 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:55.201355 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.201265 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle\") pod \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " Jun 12 18:49:55.201355 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.201349 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmm9d\" (UniqueName: \"kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d\") pod \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " Jun 12 18:49:55.201551 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.201387 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util\") pod \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\" (UID: \"654aec53-a9aa-4b87-95d3-0881ccf04fa5\") " Jun 12 18:49:55.201910 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.201876 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle" (OuterVolumeSpecName: "bundle") pod "654aec53-a9aa-4b87-95d3-0881ccf04fa5" (UID: "654aec53-a9aa-4b87-95d3-0881ccf04fa5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:49:55.203549 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.203519 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d" (OuterVolumeSpecName: "kube-api-access-pmm9d") pod "654aec53-a9aa-4b87-95d3-0881ccf04fa5" (UID: "654aec53-a9aa-4b87-95d3-0881ccf04fa5"). InnerVolumeSpecName "kube-api-access-pmm9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:49:55.206941 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.206916 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util" (OuterVolumeSpecName: "util") pod "654aec53-a9aa-4b87-95d3-0881ccf04fa5" (UID: "654aec53-a9aa-4b87-95d3-0881ccf04fa5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 18:49:55.302488 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.302420 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-util\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:49:55.302488 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.302453 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/654aec53-a9aa-4b87-95d3-0881ccf04fa5-bundle\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:49:55.302488 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.302463 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmm9d\" (UniqueName: \"kubernetes.io/projected/654aec53-a9aa-4b87-95d3-0881ccf04fa5-kube-api-access-pmm9d\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:49:55.909854 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.909824 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" Jun 12 18:49:55.909854 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.909837 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748gxx79" event={"ID":"654aec53-a9aa-4b87-95d3-0881ccf04fa5","Type":"ContainerDied","Data":"14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589"} Jun 12 18:49:55.910068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:49:55.909879 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14a5e5c1f789dacf21ee47e86f7dba7373278be0dbfa8af1807fcdc29c331589" Jun 12 18:50:44.328484 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328451 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328816 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="extract" Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328828 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="extract" Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328858 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="util" Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328863 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="util" Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328870 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="pull" Jun 12 18:50:44.328923 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328875 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="pull" Jun 12 18:50:44.329114 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.328929 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="654aec53-a9aa-4b87-95d3-0881ccf04fa5" containerName="extract" Jun 12 18:50:44.331233 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.331217 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:44.340550 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.340525 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:44.423855 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.423823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9\") pod \"authorino-5d4b67c446-pr94q\" (UID: \"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3\") " pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:44.524450 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.524419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9\") pod \"authorino-5d4b67c446-pr94q\" (UID: \"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3\") " pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:44.533684 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.533665 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9\") pod \"authorino-5d4b67c446-pr94q\" (UID: \"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3\") " pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:44.550751 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.550722 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:44.550981 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.550968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:44.804940 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.804902 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:50:44.807448 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.807429 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.813430 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.813406 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 12 18:50:44.819482 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.819455 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:50:44.826651 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.826613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz99t\" (UniqueName: \"kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.826762 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.826657 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.901464 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.901444 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:44.902632 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:50:44.902608 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c1787bd_5dc9_450e_a7d2_7e0bd63fc0d3.slice/crio-6579a8d3a05bb9462ab51e03fef49fabf3e6f7b0644868d3b037ef69bb555e4c WatchSource:0}: Error finding container 6579a8d3a05bb9462ab51e03fef49fabf3e6f7b0644868d3b037ef69bb555e4c: Status 404 returned error can't find the container with id 6579a8d3a05bb9462ab51e03fef49fabf3e6f7b0644868d3b037ef69bb555e4c Jun 12 18:50:44.927684 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.927658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz99t\" (UniqueName: \"kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.927772 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.927686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.930139 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.930118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:44.936960 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:44.936933 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz99t\" (UniqueName: \"kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t\") pod \"authorino-85994578b4-qh5z7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:45.118441 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:45.118406 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:50:45.183632 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:45.183595 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pr94q" event={"ID":"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3","Type":"ContainerStarted","Data":"6579a8d3a05bb9462ab51e03fef49fabf3e6f7b0644868d3b037ef69bb555e4c"} Jun 12 18:50:45.461497 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:45.461464 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:50:46.190812 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.190782 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85994578b4-qh5z7" event={"ID":"a7c9077b-33c8-433d-b138-d0508e32edd7","Type":"ContainerStarted","Data":"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3"} Jun 12 18:50:46.191016 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.190821 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85994578b4-qh5z7" event={"ID":"a7c9077b-33c8-433d-b138-d0508e32edd7","Type":"ContainerStarted","Data":"a34d5e17827b835ab6fa65012962f96e1f263605a137964446b259a4bc5fe10c"} Jun 12 18:50:46.192288 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.192264 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-pr94q" podUID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" containerName="authorino" containerID="cri-o://e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719" gracePeriod=30 Jun 12 18:50:46.192414 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.192281 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pr94q" event={"ID":"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3","Type":"ContainerStarted","Data":"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719"} Jun 12 18:50:46.208613 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.208567 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85994578b4-qh5z7" podStartSLOduration=1.9004400339999998 podStartE2EDuration="2.208553463s" totalImagesPullingTime="308.113429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:50:46.207599525 +0000 UTC m=+763.773876783" watchObservedRunningTime="2026-06-12 18:50:46.208553463 +0000 UTC m=+763.774830690" Jun 12 18:50:46.223217 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.223142 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-pr94q" podStartSLOduration=1.789440149 podStartE2EDuration="2.223118674s" totalImagesPullingTime="433.678525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:50:46.222735848 +0000 UTC m=+763.789013084" watchObservedRunningTime="2026-06-12 18:50:46.223118674 +0000 UTC m=+763.789395910" Jun 12 18:50:46.238526 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.238491 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:50:46.238699 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.238679 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qbz9w" podUID="95a87ec4-ba31-4198-b579-f273f0d0b18c" containerName="authorino" containerID="cri-o://f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9" gracePeriod=30 Jun 12 18:50:46.659212 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.659192 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:46.661948 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.661928 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:50:46.744996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.744921 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9\") pod \"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3\" (UID: \"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3\") " Jun 12 18:50:46.744996 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.744979 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r2bn\" (UniqueName: \"kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn\") pod \"95a87ec4-ba31-4198-b579-f273f0d0b18c\" (UID: \"95a87ec4-ba31-4198-b579-f273f0d0b18c\") " Jun 12 18:50:46.750335 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.747965 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9" (OuterVolumeSpecName: "kube-api-access-p92x9") pod "1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" (UID: "1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3"). InnerVolumeSpecName "kube-api-access-p92x9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:50:46.753069 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.753041 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn" (OuterVolumeSpecName: "kube-api-access-8r2bn") pod "95a87ec4-ba31-4198-b579-f273f0d0b18c" (UID: "95a87ec4-ba31-4198-b579-f273f0d0b18c"). InnerVolumeSpecName "kube-api-access-8r2bn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:50:46.846283 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.846258 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3-kube-api-access-p92x9\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:50:46.846283 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:46.846283 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r2bn\" (UniqueName: \"kubernetes.io/projected/95a87ec4-ba31-4198-b579-f273f0d0b18c-kube-api-access-8r2bn\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:50:47.199628 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.199596 2577 generic.go:356] "Generic (PLEG): container finished" podID="95a87ec4-ba31-4198-b579-f273f0d0b18c" containerID="f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9" exitCode=0 Jun 12 18:50:47.199820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.199653 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbz9w" Jun 12 18:50:47.199820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.199671 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbz9w" event={"ID":"95a87ec4-ba31-4198-b579-f273f0d0b18c","Type":"ContainerDied","Data":"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9"} Jun 12 18:50:47.199820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.199722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbz9w" event={"ID":"95a87ec4-ba31-4198-b579-f273f0d0b18c","Type":"ContainerDied","Data":"c5761f61cd59a592e7824317ce675b3ff2ec79f515b5152d74c29770836023b3"} Jun 12 18:50:47.199820 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.199748 2577 scope.go:117] "RemoveContainer" containerID="f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9" Jun 12 18:50:47.201226 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.201204 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" containerID="e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719" exitCode=0 Jun 12 18:50:47.201333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.201277 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pr94q" Jun 12 18:50:47.201393 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.201294 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pr94q" event={"ID":"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3","Type":"ContainerDied","Data":"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719"} Jun 12 18:50:47.201453 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.201419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pr94q" event={"ID":"1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3","Type":"ContainerDied","Data":"6579a8d3a05bb9462ab51e03fef49fabf3e6f7b0644868d3b037ef69bb555e4c"} Jun 12 18:50:47.208816 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.208797 2577 scope.go:117] "RemoveContainer" containerID="f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9" Jun 12 18:50:47.209081 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:50:47.209060 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9\": container with ID starting with f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9 not found: ID does not exist" containerID="f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9" Jun 12 18:50:47.209175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.209090 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9"} err="failed to get container status \"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9\": rpc error: code = NotFound desc = could not find container \"f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9\": container with ID starting with f8d558f51de9e8d599faa4640da2963f601cb4d1e88521f72bc50a6d6b70baf9 not found: ID does not exist" Jun 12 18:50:47.209175 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.209111 2577 scope.go:117] "RemoveContainer" containerID="e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719" Jun 12 18:50:47.217958 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.217937 2577 scope.go:117] "RemoveContainer" containerID="e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719" Jun 12 18:50:47.218202 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:50:47.218183 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719\": container with ID starting with e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719 not found: ID does not exist" containerID="e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719" Jun 12 18:50:47.218293 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.218207 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719"} err="failed to get container status \"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719\": rpc error: code = NotFound desc = could not find container \"e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719\": container with ID starting with e87ce6d7c5018d48a7e0fe15d66af14e4418be30986b9b83e318f2cf87038719 not found: ID does not exist" Jun 12 18:50:47.229169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.229149 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:50:47.232635 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.232617 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbz9w"] Jun 12 18:50:47.253794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.253775 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:47.264180 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:47.264160 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pr94q"] Jun 12 18:50:48.899947 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:48.899916 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" path="/var/lib/kubelet/pods/1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3/volumes" Jun 12 18:50:48.900339 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:48.900234 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a87ec4-ba31-4198-b579-f273f0d0b18c" path="/var/lib/kubelet/pods/95a87ec4-ba31-4198-b579-f273f0d0b18c/volumes" Jun 12 18:50:50.046354 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046307 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 18:50:50.046712 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046668 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a87ec4-ba31-4198-b579-f273f0d0b18c" containerName="authorino" Jun 12 18:50:50.046712 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046680 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a87ec4-ba31-4198-b579-f273f0d0b18c" containerName="authorino" Jun 12 18:50:50.046712 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046691 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" containerName="authorino" Jun 12 18:50:50.046712 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046697 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" containerName="authorino" Jun 12 18:50:50.046845 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046763 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c1787bd-5dc9-450e-a7d2-7e0bd63fc0d3" containerName="authorino" Jun 12 18:50:50.046845 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.046773 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a87ec4-ba31-4198-b579-f273f0d0b18c" containerName="authorino" Jun 12 18:50:50.049479 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.049454 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.052205 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.052176 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 12 18:50:50.052337 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.052263 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fcrts\"" Jun 12 18:50:50.053143 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.053126 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 12 18:50:50.058276 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.058252 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 18:50:50.072824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.072797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.072905 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.072831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9kcq\" (UniqueName: \"kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.174282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.174255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.174282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.174290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9kcq\" (UniqueName: \"kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.176545 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.176517 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.182252 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.182229 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9kcq\" (UniqueName: \"kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq\") pod \"maas-controller-7556944b6b-9q64r\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.361371 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.361287 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:50.700370 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:50.700335 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 18:50:50.701732 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:50:50.701704 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc4e717_fc04_486c_9722_547de15d98a3.slice/crio-9e34a30d27d85459b1462f0b1e99d094ede792fe70b5d8832671adadb77234cb WatchSource:0}: Error finding container 9e34a30d27d85459b1462f0b1e99d094ede792fe70b5d8832671adadb77234cb: Status 404 returned error can't find the container with id 9e34a30d27d85459b1462f0b1e99d094ede792fe70b5d8832671adadb77234cb Jun 12 18:50:51.225970 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:51.225936 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7556944b6b-9q64r" event={"ID":"5fc4e717-fc04-486c-9722-547de15d98a3","Type":"ContainerStarted","Data":"9e34a30d27d85459b1462f0b1e99d094ede792fe70b5d8832671adadb77234cb"} Jun 12 18:50:53.240793 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:53.240751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7556944b6b-9q64r" event={"ID":"5fc4e717-fc04-486c-9722-547de15d98a3","Type":"ContainerStarted","Data":"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74"} Jun 12 18:50:53.241153 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:53.240869 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:50:53.288516 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:50:53.288471 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7556944b6b-9q64r" podStartSLOduration=1.239404162 podStartE2EDuration="3.28845641s" totalImagesPullingTime="2.049052248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:50:53.287026515 +0000 UTC m=+770.853303749" watchObservedRunningTime="2026-06-12 18:50:53.28845641 +0000 UTC m=+770.854733645" Jun 12 18:51:04.252895 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:04.252833 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 18:51:24.595550 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:24.595471 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:51:32.463908 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.463875 2577 generic.go:356] "Generic (PLEG): container finished" podID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerID="ff9539ff1d2b62b2eeda9d53db5f41b2018da09e00d149f2d9a47ed422227178" exitCode=0 Jun 12 18:51:32.464570 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.463946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" event={"ID":"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7","Type":"ContainerDied","Data":"ff9539ff1d2b62b2eeda9d53db5f41b2018da09e00d149f2d9a47ed422227178"} Jun 12 18:51:32.464570 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.464281 2577 scope.go:117] "RemoveContainer" containerID="ff9539ff1d2b62b2eeda9d53db5f41b2018da09e00d149f2d9a47ed422227178" Jun 12 18:51:32.465216 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.465188 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:51:32.465333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.465252 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:51:32.465333 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:32.465282 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 12 18:51:33.471189 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:33.471151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" event={"ID":"739eb282-2e8b-4a8a-9164-52ba1fb0a2a7","Type":"ContainerStarted","Data":"d95121cb36f00a08eb097dec73bdcc66852b1a04f7f807738fbb8dc49c2cbce4"} Jun 12 18:51:33.558084 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:33.558038 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:51:33.559537 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:33.559509 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-69bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jun 12 18:51:33.559796 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:33.559757 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podUID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jun 12 18:51:34.558678 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:34.558638 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-69bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jun 12 18:51:34.559170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:34.558697 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podUID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jun 12 18:51:35.559136 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:35.559090 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-69bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jun 12 18:51:35.559603 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:35.559168 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podUID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jun 12 18:51:36.558849 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:36.558798 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-69bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jun 12 18:51:36.559043 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:36.558872 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podUID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jun 12 18:51:37.558653 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:37.558604 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-69bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jun 12 18:51:37.559149 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:37.558675 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" podUID="739eb282-2e8b-4a8a-9164-52ba1fb0a2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jun 12 18:51:37.623897 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:37.623852 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:51:38.562522 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:38.562492 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:51:38.563657 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:51:38.563637 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-69bs2" Jun 12 18:53:03.491039 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:53:03.490965 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:53:03.492651 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:53:03.492626 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:54:16.503705 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.503669 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:54:16.506373 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.506352 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.508956 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.508934 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 12 18:54:16.517551 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.517526 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:54:16.542665 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.542637 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdvs4\" (UniqueName: \"kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.542796 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.542709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.542796 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.542729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.643273 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.643239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.643273 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.643275 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.643479 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.643342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvs4\" (UniqueName: \"kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.644051 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.644032 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.645617 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.645598 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.652169 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.652148 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvs4\" (UniqueName: \"kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4\") pod \"authorino-6587cd74-tfz5d\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:16.815967 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:16.815891 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:54:17.166521 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:17.166497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:54:17.168264 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:54:17.168238 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bb9a606_703f_4ed6_a01e_5c29c675e6b6.slice/crio-57cbd0d9e48aad40f6f49c6ee8084808e412c61bbdeadfeaa20f505ef44b82ac WatchSource:0}: Error finding container 57cbd0d9e48aad40f6f49c6ee8084808e412c61bbdeadfeaa20f505ef44b82ac: Status 404 returned error can't find the container with id 57cbd0d9e48aad40f6f49c6ee8084808e412c61bbdeadfeaa20f505ef44b82ac Jun 12 18:54:17.169850 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:17.169833 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 18:54:17.382836 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:17.382803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6587cd74-tfz5d" event={"ID":"7bb9a606-703f-4ed6-a01e-5c29c675e6b6","Type":"ContainerStarted","Data":"57cbd0d9e48aad40f6f49c6ee8084808e412c61bbdeadfeaa20f505ef44b82ac"} Jun 12 18:54:18.389936 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.389892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6587cd74-tfz5d" event={"ID":"7bb9a606-703f-4ed6-a01e-5c29c675e6b6","Type":"ContainerStarted","Data":"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45"} Jun 12 18:54:18.405843 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.405788 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6587cd74-tfz5d" podStartSLOduration=2.036394905 podStartE2EDuration="2.405769386s" totalImagesPullingTime="369.374481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:54:18.405609865 +0000 UTC m=+975.971887113" watchObservedRunningTime="2026-06-12 18:54:18.405769386 +0000 UTC m=+975.972046632" Jun 12 18:54:18.437024 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.436991 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:54:18.437243 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.437219 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-85994578b4-qh5z7" podUID="a7c9077b-33c8-433d-b138-d0508e32edd7" containerName="authorino" containerID="cri-o://7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3" gracePeriod=30 Jun 12 18:54:18.816206 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.816183 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:54:18.861635 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.861608 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz99t\" (UniqueName: \"kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t\") pod \"a7c9077b-33c8-433d-b138-d0508e32edd7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " Jun 12 18:54:18.861767 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.861650 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert\") pod \"a7c9077b-33c8-433d-b138-d0508e32edd7\" (UID: \"a7c9077b-33c8-433d-b138-d0508e32edd7\") " Jun 12 18:54:18.863666 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.863634 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t" (OuterVolumeSpecName: "kube-api-access-xz99t") pod "a7c9077b-33c8-433d-b138-d0508e32edd7" (UID: "a7c9077b-33c8-433d-b138-d0508e32edd7"). InnerVolumeSpecName "kube-api-access-xz99t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:54:18.889509 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.886645 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a7c9077b-33c8-433d-b138-d0508e32edd7" (UID: "a7c9077b-33c8-433d-b138-d0508e32edd7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:54:18.962394 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.962364 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz99t\" (UniqueName: \"kubernetes.io/projected/a7c9077b-33c8-433d-b138-d0508e32edd7-kube-api-access-xz99t\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:54:18.962394 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:18.962391 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7c9077b-33c8-433d-b138-d0508e32edd7-tls-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:54:19.396702 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.396672 2577 generic.go:356] "Generic (PLEG): container finished" podID="a7c9077b-33c8-433d-b138-d0508e32edd7" containerID="7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3" exitCode=0 Jun 12 18:54:19.397170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.396721 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85994578b4-qh5z7" Jun 12 18:54:19.397170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.396754 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85994578b4-qh5z7" event={"ID":"a7c9077b-33c8-433d-b138-d0508e32edd7","Type":"ContainerDied","Data":"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3"} Jun 12 18:54:19.397170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.396797 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85994578b4-qh5z7" event={"ID":"a7c9077b-33c8-433d-b138-d0508e32edd7","Type":"ContainerDied","Data":"a34d5e17827b835ab6fa65012962f96e1f263605a137964446b259a4bc5fe10c"} Jun 12 18:54:19.397170 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.396817 2577 scope.go:117] "RemoveContainer" containerID="7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3" Jun 12 18:54:19.406183 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.406154 2577 scope.go:117] "RemoveContainer" containerID="7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3" Jun 12 18:54:19.406458 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:54:19.406440 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3\": container with ID starting with 7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3 not found: ID does not exist" containerID="7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3" Jun 12 18:54:19.406533 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.406466 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3"} err="failed to get container status \"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3\": rpc error: code = NotFound desc = could not find container \"7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3\": container with ID starting with 7fc227838c5cb1f4a70ccbd3f036774cbcd71db13a33985ed3bbeae40105f8a3 not found: ID does not exist" Jun 12 18:54:19.428553 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.428529 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:54:19.434064 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:19.434043 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85994578b4-qh5z7"] Jun 12 18:54:20.900401 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:20.900368 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7c9077b-33c8-433d-b138-d0508e32edd7" path="/var/lib/kubelet/pods/a7c9077b-33c8-433d-b138-d0508e32edd7/volumes" Jun 12 18:54:28.628991 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.628955 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8"] Jun 12 18:54:28.629419 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.629380 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7c9077b-33c8-433d-b138-d0508e32edd7" containerName="authorino" Jun 12 18:54:28.629419 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.629394 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7c9077b-33c8-433d-b138-d0508e32edd7" containerName="authorino" Jun 12 18:54:28.629529 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.629484 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7c9077b-33c8-433d-b138-d0508e32edd7" containerName="authorino" Jun 12 18:54:28.631781 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.631765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.637413 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.637384 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 12 18:54:28.637546 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.637418 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 12 18:54:28.637546 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.637457 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vz5g4\"" Jun 12 18:54:28.637546 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.637419 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 12 18:54:28.651866 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.651841 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8"] Jun 12 18:54:28.746142 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.746342 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.746342 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vd5\" (UniqueName: \"kubernetes.io/projected/cc42876b-e162-404b-b2e4-c663e9de32a3-kube-api-access-l2vd5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.746342 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.746515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746391 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.746515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.746470 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc42876b-e162-404b-b2e4-c663e9de32a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847576 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847750 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc42876b-e162-404b-b2e4-c663e9de32a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847750 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847750 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847750 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vd5\" (UniqueName: \"kubernetes.io/projected/cc42876b-e162-404b-b2e4-c663e9de32a3-kube-api-access-l2vd5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847968 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.847968 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847932 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.848075 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.847971 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.848075 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.848029 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.850033 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.850000 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc42876b-e162-404b-b2e4-c663e9de32a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.850245 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.850226 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc42876b-e162-404b-b2e4-c663e9de32a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.856543 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.856522 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vd5\" (UniqueName: \"kubernetes.io/projected/cc42876b-e162-404b-b2e4-c663e9de32a3-kube-api-access-l2vd5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8\" (UID: \"cc42876b-e162-404b-b2e4-c663e9de32a3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:28.942688 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:28.942652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:54:29.007732 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:29.007681 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:54:29.308356 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:29.308326 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8"] Jun 12 18:54:29.310381 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:54:29.310351 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc42876b_e162_404b_b2e4_c663e9de32a3.slice/crio-a561d99c4e0992a65f3aee75f0dffc1291640174d545f0d67603a41e9729f6af WatchSource:0}: Error finding container a561d99c4e0992a65f3aee75f0dffc1291640174d545f0d67603a41e9729f6af: Status 404 returned error can't find the container with id a561d99c4e0992a65f3aee75f0dffc1291640174d545f0d67603a41e9729f6af Jun 12 18:54:29.460705 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:29.460665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" event={"ID":"cc42876b-e162-404b-b2e4-c663e9de32a3","Type":"ContainerStarted","Data":"a561d99c4e0992a65f3aee75f0dffc1291640174d545f0d67603a41e9729f6af"} Jun 12 18:54:33.489909 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:33.489874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" event={"ID":"cc42876b-e162-404b-b2e4-c663e9de32a3","Type":"ContainerStarted","Data":"918f42bcff7bd0ff30aced4926b24433253e7f7f651e20dad4e924b7b3fa74ca"} Jun 12 18:54:34.359427 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:34.359396 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:54:46.761412 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.761377 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7"] Jun 12 18:54:46.764714 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.764691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.768263 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.768246 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 12 18:54:46.779264 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.779240 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7"] Jun 12 18:54:46.923363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.923363 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.923606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.923606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbf7h\" (UniqueName: \"kubernetes.io/projected/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kube-api-access-gbf7h\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.923606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:46.923606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:46.923566 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025057 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.024974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025057 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbf7h\" (UniqueName: \"kubernetes.io/projected/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kube-api-access-gbf7h\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025236 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025236 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025350 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025420 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025573 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025536 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025662 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025639 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.025725 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.025666 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.027124 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.027104 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.027550 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.027532 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.034738 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.034721 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbf7h\" (UniqueName: \"kubernetes.io/projected/07cb7e13-2b92-403d-8d8d-df5edcfc4ca7-kube-api-access-gbf7h\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7\" (UID: \"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.077703 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.077671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:54:47.428331 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.428280 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7"] Jun 12 18:54:47.429730 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:54:47.429705 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cb7e13_2b92_403d_8d8d_df5edcfc4ca7.slice/crio-b27ada46e072acb9adf0b290af37f742b9ff68b5212c3865458b6a1890dd918f WatchSource:0}: Error finding container b27ada46e072acb9adf0b290af37f742b9ff68b5212c3865458b6a1890dd918f: Status 404 returned error can't find the container with id b27ada46e072acb9adf0b290af37f742b9ff68b5212c3865458b6a1890dd918f Jun 12 18:54:47.571944 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.571901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" event={"ID":"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7","Type":"ContainerStarted","Data":"fe08c2bc011d61e3fb5eed2ee07537e919220c5e4602efb2056bb3c1c719042c"} Jun 12 18:54:47.572171 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.571963 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" event={"ID":"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7","Type":"ContainerStarted","Data":"b27ada46e072acb9adf0b290af37f742b9ff68b5212c3865458b6a1890dd918f"} Jun 12 18:54:47.653893 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:54:47.653860 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:55:05.369499 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:05.369457 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:55:20.654544 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:20.654498 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:55:27.803179 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:27.803078 2577 generic.go:356] "Generic (PLEG): container finished" podID="cc42876b-e162-404b-b2e4-c663e9de32a3" containerID="918f42bcff7bd0ff30aced4926b24433253e7f7f651e20dad4e924b7b3fa74ca" exitCode=0 Jun 12 18:55:27.803179 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:27.803168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" event={"ID":"cc42876b-e162-404b-b2e4-c663e9de32a3","Type":"ContainerDied","Data":"918f42bcff7bd0ff30aced4926b24433253e7f7f651e20dad4e924b7b3fa74ca"} Jun 12 18:55:29.817626 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:29.817587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" event={"ID":"cc42876b-e162-404b-b2e4-c663e9de32a3","Type":"ContainerStarted","Data":"812b830c3a18a606be1ff64ce519f4990d2379b3a57799a50937630ee34e27d6"} Jun 12 18:55:29.838706 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:29.838526 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" podStartSLOduration=2.966752119 podStartE2EDuration="1m1.838506783s" totalImagesPullingTime="4.82825926s" totalInitContainerRuntime="54.043495404s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:54:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:55:29.837818036 +0000 UTC m=+1047.404095307" watchObservedRunningTime="2026-06-12 18:55:29.838506783 +0000 UTC m=+1047.404784019" Jun 12 18:55:38.943786 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.943748 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:55:38.944282 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.943831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:55:38.957045 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.957003 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:55:38.970635 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.970598 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf"] Jun 12 18:55:38.974369 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.974334 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:38.977627 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:38.977590 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 12 18:55:39.003515 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.003478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf"] Jun 12 18:55:39.021660 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.021624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.021851 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.021675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.021851 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.021772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58pw7\" (UniqueName: \"kubernetes.io/projected/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kube-api-access-58pw7\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.021959 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.021920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.022056 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.022037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.022107 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.022091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cf3f2c-838d-4191-b0dd-a1c082ef797b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122783 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cf3f2c-838d-4191-b0dd-a1c082ef797b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58pw7\" (UniqueName: \"kubernetes.io/projected/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kube-api-access-58pw7\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.122987 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.122964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.123284 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.123218 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.123345 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.123284 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.123345 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.123326 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.125161 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.125138 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cf3f2c-838d-4191-b0dd-a1c082ef797b-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.125242 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.125225 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cf3f2c-838d-4191-b0dd-a1c082ef797b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.131969 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.131944 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58pw7\" (UniqueName: \"kubernetes.io/projected/77cf3f2c-838d-4191-b0dd-a1c082ef797b-kube-api-access-58pw7\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-plckf\" (UID: \"77cf3f2c-838d-4191-b0dd-a1c082ef797b\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.284573 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.284471 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:55:39.643292 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.643244 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf"] Jun 12 18:55:39.648619 ip-10-0-141-141 kubenswrapper[2577]: W0612 18:55:39.648589 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77cf3f2c_838d_4191_b0dd_a1c082ef797b.slice/crio-6ce1e5b6e518a6c3685d8c1946e363402e27304c4bb6089b5083bc36b001060e WatchSource:0}: Error finding container 6ce1e5b6e518a6c3685d8c1946e363402e27304c4bb6089b5083bc36b001060e: Status 404 returned error can't find the container with id 6ce1e5b6e518a6c3685d8c1946e363402e27304c4bb6089b5083bc36b001060e Jun 12 18:55:39.878750 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.878704 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" event={"ID":"77cf3f2c-838d-4191-b0dd-a1c082ef797b","Type":"ContainerStarted","Data":"8a2936f9c7e2ec89e52e5f0516f2f67b62b68aea5163110fb9484c64e7ebd998"} Jun 12 18:55:39.879005 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.878767 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" event={"ID":"77cf3f2c-838d-4191-b0dd-a1c082ef797b","Type":"ContainerStarted","Data":"6ce1e5b6e518a6c3685d8c1946e363402e27304c4bb6089b5083bc36b001060e"} Jun 12 18:55:39.890980 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:39.890949 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8" Jun 12 18:55:40.250822 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:40.250786 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:55:56.981772 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:56.981740 2577 generic.go:356] "Generic (PLEG): container finished" podID="07cb7e13-2b92-403d-8d8d-df5edcfc4ca7" containerID="fe08c2bc011d61e3fb5eed2ee07537e919220c5e4602efb2056bb3c1c719042c" exitCode=0 Jun 12 18:55:56.982201 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:56.981813 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" event={"ID":"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7","Type":"ContainerDied","Data":"fe08c2bc011d61e3fb5eed2ee07537e919220c5e4602efb2056bb3c1c719042c"} Jun 12 18:55:57.989824 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:57.989787 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" event={"ID":"07cb7e13-2b92-403d-8d8d-df5edcfc4ca7","Type":"ContainerStarted","Data":"036229b351617d0f2bd46fbc0efff04d605a2da5bebf48917bbaf6f362d78be5"} Jun 12 18:55:58.011068 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:55:58.011010 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" podStartSLOduration=3.129612595 podStartE2EDuration="1m12.010993373s" totalImagesPullingTime="200.651498ms" totalInitContainerRuntime="1m8.68072928s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:55:58.009258546 +0000 UTC m=+1075.575535782" watchObservedRunningTime="2026-06-12 18:55:58.010993373 +0000 UTC m=+1075.577270608" Jun 12 18:56:07.077978 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:56:07.077929 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:56:07.078385 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:56:07.077994 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:56:07.090487 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:56:07.090461 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:56:08.059595 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:56:08.059562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7" Jun 12 18:58:03.538104 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:03.538075 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:58:03.540827 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:03.540804 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 18:58:40.910637 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:40.910604 2577 generic.go:356] "Generic (PLEG): container finished" podID="77cf3f2c-838d-4191-b0dd-a1c082ef797b" containerID="8a2936f9c7e2ec89e52e5f0516f2f67b62b68aea5163110fb9484c64e7ebd998" exitCode=0 Jun 12 18:58:40.911052 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:40.910671 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" event={"ID":"77cf3f2c-838d-4191-b0dd-a1c082ef797b","Type":"ContainerDied","Data":"8a2936f9c7e2ec89e52e5f0516f2f67b62b68aea5163110fb9484c64e7ebd998"} Jun 12 18:58:41.918941 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:41.918909 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" event={"ID":"77cf3f2c-838d-4191-b0dd-a1c082ef797b","Type":"ContainerStarted","Data":"1fed2d9153a797c2fd25b98f15970457cd096c297d03ff78860470f670111bcb"} Jun 12 18:58:41.939164 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:41.939103 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" podStartSLOduration=3.190160661 podStartE2EDuration="3m3.939086588s" totalImagesPullingTime="230.015584ms" totalInitContainerRuntime="3m0.518910343s" isStatefulPod=false podCreationTimestamp="2026-06-12 18:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 18:58:41.937372541 +0000 UTC m=+1239.503649788" watchObservedRunningTime="2026-06-12 18:58:41.939086588 +0000 UTC m=+1239.505363822" Jun 12 18:58:49.284916 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:49.284881 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:58:49.284916 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:49.284926 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:58:49.297567 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:49.297543 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:58:49.972153 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:49.972123 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-plckf" Jun 12 18:58:53.019609 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.019525 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:58:53.020070 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.019704 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6587cd74-tfz5d" podUID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" containerName="authorino" containerID="cri-o://f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45" gracePeriod=30 Jun 12 18:58:53.403952 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.403928 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:58:53.483774 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.483743 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert\") pod \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " Jun 12 18:58:53.483943 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.483791 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdvs4\" (UniqueName: \"kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4\") pod \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " Jun 12 18:58:53.483943 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.483866 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca\") pod \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\" (UID: \"7bb9a606-703f-4ed6-a01e-5c29c675e6b6\") " Jun 12 18:58:53.485906 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.485866 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4" (OuterVolumeSpecName: "kube-api-access-rdvs4") pod "7bb9a606-703f-4ed6-a01e-5c29c675e6b6" (UID: "7bb9a606-703f-4ed6-a01e-5c29c675e6b6"). InnerVolumeSpecName "kube-api-access-rdvs4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 18:58:53.490606 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.490578 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "7bb9a606-703f-4ed6-a01e-5c29c675e6b6" (UID: "7bb9a606-703f-4ed6-a01e-5c29c675e6b6"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 18:58:53.494215 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.494194 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7bb9a606-703f-4ed6-a01e-5c29c675e6b6" (UID: "7bb9a606-703f-4ed6-a01e-5c29c675e6b6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 18:58:53.585486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.585456 2577 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-oidc-ca\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:58:53.585486 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.585483 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-tls-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:58:53.585698 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.585492 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdvs4\" (UniqueName: \"kubernetes.io/projected/7bb9a606-703f-4ed6-a01e-5c29c675e6b6-kube-api-access-rdvs4\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 18:58:53.986614 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.986579 2577 generic.go:356] "Generic (PLEG): container finished" podID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" containerID="f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45" exitCode=0 Jun 12 18:58:53.986794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.986616 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6587cd74-tfz5d" Jun 12 18:58:53.986794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.986656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6587cd74-tfz5d" event={"ID":"7bb9a606-703f-4ed6-a01e-5c29c675e6b6","Type":"ContainerDied","Data":"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45"} Jun 12 18:58:53.986794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.986695 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6587cd74-tfz5d" event={"ID":"7bb9a606-703f-4ed6-a01e-5c29c675e6b6","Type":"ContainerDied","Data":"57cbd0d9e48aad40f6f49c6ee8084808e412c61bbdeadfeaa20f505ef44b82ac"} Jun 12 18:58:53.986794 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.986712 2577 scope.go:117] "RemoveContainer" containerID="f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45" Jun 12 18:58:53.998067 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.998042 2577 scope.go:117] "RemoveContainer" containerID="f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45" Jun 12 18:58:53.998407 ip-10-0-141-141 kubenswrapper[2577]: E0612 18:58:53.998378 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45\": container with ID starting with f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45 not found: ID does not exist" containerID="f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45" Jun 12 18:58:53.998585 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:53.998558 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45"} err="failed to get container status \"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45\": rpc error: code = NotFound desc = could not find container \"f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45\": container with ID starting with f4d0c690e3fba12baaefcdd8d599a89dcedf039e1f43d01e39565f1db1e1ac45 not found: ID does not exist" Jun 12 18:58:54.016730 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:54.016707 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:58:54.021284 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:54.021261 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6587cd74-tfz5d"] Jun 12 18:58:54.900713 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:58:54.900677 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" path="/var/lib/kubelet/pods/7bb9a606-703f-4ed6-a01e-5c29c675e6b6/volumes" Jun 12 18:59:23.159866 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:59:23.159816 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:59:33.752012 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:59:33.751973 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:59:42.350816 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:59:42.350781 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 18:59:52.457545 ip-10-0-141-141 kubenswrapper[2577]: I0612 18:59:52.457508 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:00:01.751414 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:01.751380 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:00:12.146902 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:12.146867 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:00:19.960202 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:19.960165 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 19:00:19.960741 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:19.960406 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7556944b6b-9q64r" podUID="5fc4e717-fc04-486c-9722-547de15d98a3" containerName="manager" containerID="cri-o://0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74" gracePeriod=30 Jun 12 19:00:20.329526 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.329500 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 19:00:20.459364 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.459333 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9kcq\" (UniqueName: \"kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq\") pod \"5fc4e717-fc04-486c-9722-547de15d98a3\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " Jun 12 19:00:20.459526 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.459411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert\") pod \"5fc4e717-fc04-486c-9722-547de15d98a3\" (UID: \"5fc4e717-fc04-486c-9722-547de15d98a3\") " Jun 12 19:00:20.461530 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.461504 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5fc4e717-fc04-486c-9722-547de15d98a3" (UID: "5fc4e717-fc04-486c-9722-547de15d98a3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 19:00:20.461530 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.461515 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq" (OuterVolumeSpecName: "kube-api-access-q9kcq") pod "5fc4e717-fc04-486c-9722-547de15d98a3" (UID: "5fc4e717-fc04-486c-9722-547de15d98a3"). InnerVolumeSpecName "kube-api-access-q9kcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 19:00:20.498856 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.498825 2577 generic.go:356] "Generic (PLEG): container finished" podID="5fc4e717-fc04-486c-9722-547de15d98a3" containerID="0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74" exitCode=0 Jun 12 19:00:20.498977 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.498865 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7556944b6b-9q64r" Jun 12 19:00:20.498977 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.498875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7556944b6b-9q64r" event={"ID":"5fc4e717-fc04-486c-9722-547de15d98a3","Type":"ContainerDied","Data":"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74"} Jun 12 19:00:20.498977 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.498927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7556944b6b-9q64r" event={"ID":"5fc4e717-fc04-486c-9722-547de15d98a3","Type":"ContainerDied","Data":"9e34a30d27d85459b1462f0b1e99d094ede792fe70b5d8832671adadb77234cb"} Jun 12 19:00:20.498977 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.498958 2577 scope.go:117] "RemoveContainer" containerID="0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74" Jun 12 19:00:20.508207 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.508190 2577 scope.go:117] "RemoveContainer" containerID="0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74" Jun 12 19:00:20.508478 ip-10-0-141-141 kubenswrapper[2577]: E0612 19:00:20.508457 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74\": container with ID starting with 0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74 not found: ID does not exist" containerID="0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74" Jun 12 19:00:20.508529 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.508492 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74"} err="failed to get container status \"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74\": rpc error: code = NotFound desc = could not find container \"0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74\": container with ID starting with 0d967a1016fafc2fc70200cc8f9db1d98ffba7c08c6719d82381f0d9f84ebf74 not found: ID does not exist" Jun 12 19:00:20.520974 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.520952 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 19:00:20.524432 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.524410 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7556944b6b-9q64r"] Jun 12 19:00:20.561009 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.560937 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fc4e717-fc04-486c-9722-547de15d98a3-webhook-cert\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 19:00:20.561009 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.560966 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9kcq\" (UniqueName: \"kubernetes.io/projected/5fc4e717-fc04-486c-9722-547de15d98a3-kube-api-access-q9kcq\") on node \"ip-10-0-141-141.ec2.internal\" DevicePath \"\"" Jun 12 19:00:20.900385 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:00:20.900355 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc4e717-fc04-486c-9722-547de15d98a3" path="/var/lib/kubelet/pods/5fc4e717-fc04-486c-9722-547de15d98a3/volumes" Jun 12 19:01:24.655329 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:01:24.655279 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:01:40.846180 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:01:40.846145 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:02:19.542723 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:02:19.542646 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:02:36.044634 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:02:36.044601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:02:50.450476 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:02:50.450443 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:03:03.577220 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:03:03.577189 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:03:03.581938 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:03:03.581910 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:03:06.758116 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:03:06.758083 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:04:07.148223 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:04:07.148186 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:04:16.160889 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:04:16.160850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:04:32.844563 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:04:32.844519 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:04:41.443401 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:04:41.443348 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:04:58.759219 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:04:58.759135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:05:06.956678 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:05:06.956635 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:05:39.605288 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:05:39.605251 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:05:47.652907 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:05:47.652872 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:05:51.168096 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:05:51.168058 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:05:57.419141 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:05:57.419097 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:06:04.862367 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:06:04.862330 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:06:13.457277 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:06:13.457239 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:06:38.888079 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:06:38.887992 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:06:50.455122 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:06:50.455075 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:07:39.010875 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:07:39.010841 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:07:47.466957 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:07:47.466918 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:03.621496 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:03.621463 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:08:03.626673 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:03.626651 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:08:05.959073 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:05.959028 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:14.448925 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:14.448891 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:23.457015 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:23.456971 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:32.164098 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:32.164059 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:41.165326 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:41.165273 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:50.043766 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:50.043725 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:08:58.651740 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:08:58.651704 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:09:07.154369 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:09:07.154331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:09:15.348684 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:09:15.348642 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:09:23.660692 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:09:23.660619 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:11:47.452439 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:11:47.452400 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:11:51.745105 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:11:51.745067 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:17.267451 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:17.267408 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:21.652536 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:21.652447 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:29.165007 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:29.164971 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:39.653279 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:39.653236 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:48.656863 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:48.656820 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:12:59.443488 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:12:59.443446 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:13:03.661143 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:03.661108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:13:03.667809 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:03.667783 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:13:08.060714 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:08.060671 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:13:17.950183 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:17.950140 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:13:27.381450 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:27.381402 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:13:38.179346 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:38.179294 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:13:47.150720 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:13:47.150680 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:14:19.971743 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:14:19.971652 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:03.162388 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:03.162352 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:11.556286 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:11.556242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:20.398829 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:20.398790 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:29.181594 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:29.181508 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:37.457625 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:37.457589 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:50.059996 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:50.059966 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:15:58.960075 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:15:58.960029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:04.447898 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:04.447854 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:11.366335 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:11.366260 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:20.051488 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:20.051450 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:27.760613 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:27.760574 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:38.256760 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:38.256726 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:16:56.463406 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:16:56.463329 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:05.049513 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:05.049482 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:13.458116 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:13.458076 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:21.659654 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:21.659615 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:38.961205 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:38.961171 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:47.647850 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:47.647802 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:17:56.163636 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:17:56.163601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:03.701763 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:03.701729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:18:03.710743 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:03.710713 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:18:04.657242 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:04.657210 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:13.452952 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:13.452910 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:22.148785 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:22.148703 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:31.085640 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:31.085599 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:41.953926 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:41.953892 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:18:51.250565 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:18:51.250531 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:02.253265 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:02.253225 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:11.266037 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:11.265999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:16.652820 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:16.652789 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:23.560244 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:23.560206 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:28.755489 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:28.755454 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:43.453450 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:43.453414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:19:52.156699 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:19:52.156618 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:20:01.653123 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:20:01.653082 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:20:07.448848 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:20:07.448806 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:20:31.450116 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:20:31.450080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:20:43.752402 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:20:43.752364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-26tc6"] Jun 12 19:21:22.318556 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:22.318459 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-55f6479cff-fhj7l_0aa01331-abd9-4283-9c3c-a906d359a87a/nginx/0.log" Jun 12 19:21:23.466614 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:23.466577 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc6c5cc78-fxsm6_f797a377-c317-482b-8fd3-2492eb270c16/manager/0.log" Jun 12 19:21:23.473194 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:23.473166 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc6c5cc78-fxsm6_f797a377-c317-482b-8fd3-2492eb270c16/copy-manifests/0.log" Jun 12 19:21:24.481887 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.481856 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/util/0.log" Jun 12 19:21:24.488714 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.488688 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/pull/0.log" Jun 12 19:21:24.495776 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.495753 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/extract/0.log" Jun 12 19:21:24.613469 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.613435 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/util/0.log" Jun 12 19:21:24.623795 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.623771 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/pull/0.log" Jun 12 19:21:24.631657 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.631632 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/extract/0.log" Jun 12 19:21:24.743884 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.743802 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/extract/0.log" Jun 12 19:21:24.750169 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.750146 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/util/0.log" Jun 12 19:21:24.758733 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.758712 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/pull/0.log" Jun 12 19:21:24.869256 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.869228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/extract/0.log" Jun 12 19:21:24.875972 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.875947 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/util/0.log" Jun 12 19:21:24.882685 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:24.882635 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/pull/0.log" Jun 12 19:21:25.124248 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:25.124164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5tndr_222b1e9d-e3b7-41e7-aa55-07305ea343e8/manager/0.log" Jun 12 19:21:25.455904 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:25.455864 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h2lrz_b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0/registry-server/0.log" Jun 12 19:21:25.684780 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:25.684754 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-26tc6_ff21b7d8-4915-4af4-9946-1b0bf4781155/limitador/0.log" Jun 12 19:21:26.143706 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:26.143663 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795h475q_049593e3-ea58-463a-9f0d-fd5c3557f8ba/istio-proxy/0.log" Jun 12 19:21:26.612225 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:26.612143 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/1.log" Jun 12 19:21:26.957326 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:26.957295 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-746857f878-zdttq_019abd06-072a-4108-af28-ad66c55bbb36/router/0.log" Jun 12 19:21:27.291680 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.291584 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8_cc42876b-e162-404b-b2e4-c663e9de32a3/storage-initializer/0.log" Jun 12 19:21:27.300147 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.300121 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-7nfx8_cc42876b-e162-404b-b2e4-c663e9de32a3/main/0.log" Jun 12 19:21:27.532656 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.532618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7_07cb7e13-2b92-403d-8d8d-df5edcfc4ca7/storage-initializer/0.log" Jun 12 19:21:27.540498 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.540472 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7stz7_07cb7e13-2b92-403d-8d8d-df5edcfc4ca7/main/0.log" Jun 12 19:21:27.888666 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.888637 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-plckf_77cf3f2c-838d-4191-b0dd-a1c082ef797b/storage-initializer/0.log" Jun 12 19:21:27.897190 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:27.897163 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-plckf_77cf3f2c-838d-4191-b0dd-a1c082ef797b/main/0.log" Jun 12 19:21:34.780833 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:34.780779 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dkcrw_4e14573a-c737-43c5-bd00-07679062a688/global-pull-secret-syncer/0.log" Jun 12 19:21:34.954164 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:34.954135 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8rl5k_b6661090-b2a0-4017-8029-bf18a56e6039/konnectivity-agent/0.log" Jun 12 19:21:35.038089 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:35.037998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-141.ec2.internal_4ff5b698212972d9b24580e1a2f13644/haproxy/0.log" Jun 12 19:21:39.324151 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.324114 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/extract/0.log" Jun 12 19:21:39.386067 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.386037 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/util/0.log" Jun 12 19:21:39.408661 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.408633 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595m666_1a0457de-6a0b-4030-9568-9c98103d4353/pull/0.log" Jun 12 19:21:39.442018 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.441993 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/extract/0.log" Jun 12 19:21:39.466886 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.466856 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/util/0.log" Jun 12 19:21:39.496053 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.496024 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c54kj_7d50de42-2e58-4a9f-86ff-dda2c0dfa9f2/pull/0.log" Jun 12 19:21:39.529374 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.529336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/extract/0.log" Jun 12 19:21:39.555721 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.555691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/util/0.log" Jun 12 19:21:39.588777 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.588691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gv2sl_17f3b8b0-cbcd-400a-88ea-0a17211c93d7/pull/0.log" Jun 12 19:21:39.618101 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.618071 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/extract/0.log" Jun 12 19:21:39.640837 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.640809 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/util/0.log" Jun 12 19:21:39.669817 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:39.669789 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef176c42_a65fafe6-4fce-4cfd-a9b9-63da7d298fec/pull/0.log" Jun 12 19:21:40.012301 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:40.012273 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5tndr_222b1e9d-e3b7-41e7-aa55-07305ea343e8/manager/0.log" Jun 12 19:21:40.158153 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:40.158123 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h2lrz_b5fcbbeb-48be-4153-b81d-77d2f8b8d1c0/registry-server/0.log" Jun 12 19:21:40.245335 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:40.245285 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-26tc6_ff21b7d8-4915-4af4-9946-1b0bf4781155/limitador/0.log" Jun 12 19:21:41.930434 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:41.930407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/alertmanager/0.log" Jun 12 19:21:41.952053 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:41.952027 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/config-reloader/0.log" Jun 12 19:21:41.973514 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:41.973487 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/kube-rbac-proxy-web/0.log" Jun 12 19:21:41.994247 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:41.994220 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/kube-rbac-proxy/0.log" Jun 12 19:21:42.018145 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.018117 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/kube-rbac-proxy-metric/0.log" Jun 12 19:21:42.046873 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.046849 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/prom-label-proxy/0.log" Jun 12 19:21:42.071639 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.071613 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_04adeeed-3bb7-4f07-8713-c0b0e9f711aa/init-config-reloader/0.log" Jun 12 19:21:42.120703 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.120672 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-d4vg4_aff6bbf7-a1a9-42b3-aa6f-36331e58d629/cluster-monitoring-operator/0.log" Jun 12 19:21:42.245468 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.245381 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-dbb6b689d-dcs28_ddc1a0b5-3c29-4316-91e9-9fcb1ed46980/metrics-server/0.log" Jun 12 19:21:42.272293 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.272265 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-t6qhv_5e187eea-f7e6-4123-941c-c776e0857db7/monitoring-plugin/0.log" Jun 12 19:21:42.478651 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.478614 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rm828_42129e6f-63c9-49da-b4e9-1915e1bd8b30/node-exporter/0.log" Jun 12 19:21:42.502961 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.502872 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rm828_42129e6f-63c9-49da-b4e9-1915e1bd8b30/kube-rbac-proxy/0.log" Jun 12 19:21:42.525100 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.525071 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rm828_42129e6f-63c9-49da-b4e9-1915e1bd8b30/init-textfile/0.log" Jun 12 19:21:42.553223 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.553198 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-8qw99_90dbec6c-b06a-4415-b931-9c568e40f8b2/kube-rbac-proxy-main/0.log" Jun 12 19:21:42.577042 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.577019 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-8qw99_90dbec6c-b06a-4415-b931-9c568e40f8b2/kube-rbac-proxy-self/0.log" Jun 12 19:21:42.603456 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:42.603421 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-8qw99_90dbec6c-b06a-4415-b931-9c568e40f8b2/openshift-state-metrics/0.log" Jun 12 19:21:45.224414 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:45.224384 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/1.log" Jun 12 19:21:45.233672 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:45.233640 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-8ngbd_987a55ab-eb39-48f6-886d-780f76778c0c/console-operator/2.log" Jun 12 19:21:45.822897 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:45.822867 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-784bdf947d-86wfv_5b593e2c-39e2-48ae-8d57-25c803fc79aa/console/0.log" Jun 12 19:21:46.285093 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285062 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m"] Jun 12 19:21:46.285485 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285470 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc4e717-fc04-486c-9722-547de15d98a3" containerName="manager" Jun 12 19:21:46.285533 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285486 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc4e717-fc04-486c-9722-547de15d98a3" containerName="manager" Jun 12 19:21:46.285533 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285499 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" containerName="authorino" Jun 12 19:21:46.285533 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285505 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" containerName="authorino" Jun 12 19:21:46.285630 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285582 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb9a606-703f-4ed6-a01e-5c29c675e6b6" containerName="authorino" Jun 12 19:21:46.285630 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.285591 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc4e717-fc04-486c-9722-547de15d98a3" containerName="manager" Jun 12 19:21:46.288626 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.288610 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.291360 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.291341 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g556j\"/\"kube-root-ca.crt\"" Jun 12 19:21:46.291513 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.291378 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g556j\"/\"default-dockercfg-ltdpj\"" Jun 12 19:21:46.292420 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.292401 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g556j\"/\"openshift-service-ca.crt\"" Jun 12 19:21:46.299503 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.299478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m"] Jun 12 19:21:46.323902 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.323871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-podres\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.324049 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.323919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-proc\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.324049 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.323988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-lib-modules\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.324049 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.324025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/02868383-27e2-4810-af5b-5bc95cc22e36-kube-api-access-kqx8l\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.324157 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.324102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-sys\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.424818 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-sys\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-podres\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-proc\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-lib-modules\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424909 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-sys\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/02868383-27e2-4810-af5b-5bc95cc22e36-kube-api-access-kqx8l\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425012 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.424999 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-proc\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425240 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.425039 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-lib-modules\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.425240 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.425039 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02868383-27e2-4810-af5b-5bc95cc22e36-podres\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.433719 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.433690 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/02868383-27e2-4810-af5b-5bc95cc22e36-kube-api-access-kqx8l\") pod \"perf-node-gather-daemonset-5gx4m\" (UID: \"02868383-27e2-4810-af5b-5bc95cc22e36\") " pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.598900 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.598812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:46.621300 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.621268 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-bd4ccd799-9dnkc_711b930d-d2b7-4af5-be1d-a8dc2550462f/volume-data-source-validator/0.log" Jun 12 19:21:46.945340 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.945298 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m"] Jun 12 19:21:46.947947 ip-10-0-141-141 kubenswrapper[2577]: W0612 19:21:46.947922 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod02868383_27e2_4810_af5b_5bc95cc22e36.slice/crio-cd269c4b38eb8cd90fb96d8571d7102675f77ba859c65a99ea38797eca203fbf WatchSource:0}: Error finding container cd269c4b38eb8cd90fb96d8571d7102675f77ba859c65a99ea38797eca203fbf: Status 404 returned error can't find the container with id cd269c4b38eb8cd90fb96d8571d7102675f77ba859c65a99ea38797eca203fbf Jun 12 19:21:46.955704 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:46.955677 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" event={"ID":"02868383-27e2-4810-af5b-5bc95cc22e36","Type":"ContainerStarted","Data":"cd269c4b38eb8cd90fb96d8571d7102675f77ba859c65a99ea38797eca203fbf"} Jun 12 19:21:47.963018 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:47.962975 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" event={"ID":"02868383-27e2-4810-af5b-5bc95cc22e36","Type":"ContainerStarted","Data":"25be78de761a27429d8f714a55beb048feb4a1cbff1c737f6469b6365eb92e10"} Jun 12 19:21:47.963455 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:47.963042 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:47.979713 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:47.979668 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" podStartSLOduration=1.979653725 podStartE2EDuration="1.979653725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 19:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 19:21:47.979183564 +0000 UTC m=+2625.545460799" watchObservedRunningTime="2026-06-12 19:21:47.979653725 +0000 UTC m=+2625.545930960" Jun 12 19:21:47.982117 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:47.982089 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zjnct_5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9/dns/0.log" Jun 12 19:21:48.001525 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:48.001499 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zjnct_5f2799d1-8e1e-4572-9ca2-c3453f3ec3b9/kube-rbac-proxy/0.log" Jun 12 19:21:48.057682 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:48.057646 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hc62t_280a69e5-2a94-475f-9444-3b61d6fa80a9/dns-node-resolver/0.log" Jun 12 19:21:48.724332 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:48.724282 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z67bx_9e5ce852-849e-4d39-bcbf-d6d44a3bdc36/node-ca/0.log" Jun 12 19:21:49.709294 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:49.709259 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795h475q_049593e3-ea58-463a-9f0d-fd5c3557f8ba/istio-proxy/0.log" Jun 12 19:21:49.854224 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:49.854189 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/0.log" Jun 12 19:21:49.916238 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:49.916204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/1.log" Jun 12 19:21:50.029394 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:50.029296 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-746857f878-zdttq_019abd06-072a-4108-af28-ad66c55bbb36/router/0.log" Jun 12 19:21:50.650401 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:50.650374 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-477dk_ffbd7381-8ee6-45e0-b64e-011e660db70a/serve-healthcheck-canary/0.log" Jun 12 19:21:51.412656 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:51.412625 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qm4m2_2e1989e3-9399-457d-99be-b16e05f27d91/kube-rbac-proxy/0.log" Jun 12 19:21:51.434447 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:51.434413 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qm4m2_2e1989e3-9399-457d-99be-b16e05f27d91/exporter/0.log" Jun 12 19:21:51.457928 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:51.457902 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qm4m2_2e1989e3-9399-457d-99be-b16e05f27d91/extractor/0.log" Jun 12 19:21:53.919210 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:53.919180 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-55f6479cff-fhj7l_0aa01331-abd9-4283-9c3c-a906d359a87a/nginx/0.log" Jun 12 19:21:53.984509 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:53.984477 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g556j/perf-node-gather-daemonset-5gx4m" Jun 12 19:21:54.326056 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:54.325972 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc6c5cc78-fxsm6_f797a377-c317-482b-8fd3-2492eb270c16/manager/0.log" Jun 12 19:21:54.345733 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:54.345711 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc6c5cc78-fxsm6_f797a377-c317-482b-8fd3-2492eb270c16/copy-manifests/0.log" Jun 12 19:21:56.083363 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:56.083333 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-576ccf6f5f-825dd_74346848-57dd-476a-8d80-27ddce1b1db9/manager/0.log" Jun 12 19:21:56.143038 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:21:56.143007 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zmttp_ae2816e3-d02e-49f0-ba61-3d3e804c6631/openshift-lws-operator/0.log" Jun 12 19:22:00.297842 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:00.297810 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795h475q_049593e3-ea58-463a-9f0d-fd5c3557f8ba/istio-proxy/0.log" Jun 12 19:22:00.405120 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:00.405083 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/0.log" Jun 12 19:22:00.471131 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:00.471088 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/1.log" Jun 12 19:22:00.554543 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:00.554451 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-746857f878-zdttq_019abd06-072a-4108-af28-ad66c55bbb36/router/0.log" Jun 12 19:22:01.710527 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:01.710496 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795h475q_049593e3-ea58-463a-9f0d-fd5c3557f8ba/istio-proxy/0.log" Jun 12 19:22:01.834893 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:01.834857 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/0.log" Jun 12 19:22:01.892513 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:01.892466 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/1.log" Jun 12 19:22:01.998908 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:01.998800 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-746857f878-zdttq_019abd06-072a-4108-af28-ad66c55bbb36/router/0.log" Jun 12 19:22:03.709895 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:03.709848 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795h475q_049593e3-ea58-463a-9f0d-fd5c3557f8ba/istio-proxy/0.log" Jun 12 19:22:03.801705 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:03.801674 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/0.log" Jun 12 19:22:03.866253 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:03.866211 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-69bs2_739eb282-2e8b-4a8a-9164-52ba1fb0a2a7/istio-proxy/1.log" Jun 12 19:22:03.951611 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:03.951578 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-746857f878-zdttq_019abd06-072a-4108-af28-ad66c55bbb36/router/0.log" Jun 12 19:22:04.024697 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.024622 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/kube-multus-additional-cni-plugins/0.log" Jun 12 19:22:04.047209 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.047183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/egress-router-binary-copy/0.log" Jun 12 19:22:04.072542 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.072519 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/cni-plugins/0.log" Jun 12 19:22:04.093793 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.093773 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/bond-cni-plugin/0.log" Jun 12 19:22:04.116220 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.116196 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/routeoverride-cni/0.log" Jun 12 19:22:04.141783 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.141758 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/whereabouts-cni-bincopy/0.log" Jun 12 19:22:04.163921 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.163902 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdh88_96fd2d13-f5b8-4cc1-9192-a90e13ba3df1/whereabouts-cni/0.log" Jun 12 19:22:04.236821 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.236792 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s9hn6_92c90488-bdc9-427f-b43c-e3dec7f332e2/kube-multus/0.log" Jun 12 19:22:04.318406 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.318305 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8k97v_dd0084c5-7e69-4534-8301-20b771c71427/network-metrics-daemon/0.log" Jun 12 19:22:04.336795 ip-10-0-141-141 kubenswrapper[2577]: I0612 19:22:04.336763 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8k97v_dd0084c5-7e69-4534-8301-20b771c71427/kube-rbac-proxy/0.log"