Jul 06 19:21:29.590972 ip-10-0-133-207 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 19:21:29.590981 ip-10-0-133-207 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 19:21:29.590988 ip-10-0-133-207 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 19:21:29.591195 ip-10-0-133-207 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 19:21:39.603258 ip-10-0-133-207 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 19:21:39.603273 ip-10-0-133-207 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 98510a96b1484e3685ec0766b340bbe0 -- Jul 06 19:23:38.394891 ip-10-0-133-207 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:23:38.715687 ip-10-0-133-207 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:38.715687 ip-10-0-133-207 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:23:38.715687 ip-10-0-133-207 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:38.715687 ip-10-0-133-207 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:23:38.715687 ip-10-0-133-207 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:38.717220 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.717141 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:23:38.722221 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722206 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.722221 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722221 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722226 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722229 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722231 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722234 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722236 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722239 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722241 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722244 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722246 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722248 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722251 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722253 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722256 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722259 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722261 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722264 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722267 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722270 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722273 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.722286 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722275 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722279 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722281 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722284 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722286 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722289 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722291 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722294 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722297 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722300 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722302 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722305 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722307 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722312 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722316 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722319 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722322 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722325 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722328 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722330 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.722762 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722333 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722336 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722338 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722341 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722343 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722348 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722351 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722354 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722356 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722359 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722361 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722364 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722366 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722370 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722374 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722377 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722380 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722383 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722386 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.723289 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722388 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722391 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722393 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722396 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722398 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722401 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722403 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722406 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722408 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722411 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722413 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722415 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722418 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722420 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722423 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722425 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722428 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722430 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722434 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722436 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.723756 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722439 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722441 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722444 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722446 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722448 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722451 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722454 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722458 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722460 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722463 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722466 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722468 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722470 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722473 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722475 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722478 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722480 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722483 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.722485 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723179 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.724269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723186 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723188 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723191 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723194 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723197 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723200 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723203 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723206 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723209 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723211 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723214 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723216 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723218 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723221 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723223 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723226 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723228 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723231 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723234 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723237 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.724767 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723240 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723242 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723245 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723247 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723249 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723252 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723255 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723257 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723259 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723262 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723264 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723266 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723269 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723271 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723274 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723276 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723279 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723281 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723284 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723286 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.725269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723289 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723291 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723294 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723296 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723299 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723302 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723305 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723309 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723312 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723316 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723319 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723323 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723326 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723328 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723331 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723333 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723336 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723338 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723340 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723343 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.725766 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723345 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723348 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723350 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723353 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723355 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723358 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723360 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723363 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723367 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723369 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723373 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723376 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723378 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723381 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723383 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723385 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723388 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723390 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723392 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.726269 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723395 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723397 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723401 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723404 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723408 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723411 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723414 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723416 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723418 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723421 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723424 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723426 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723429 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723432 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723434 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723436 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723439 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723441 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723444 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723810 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.726738 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723815 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723818 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723820 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723823 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723826 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723829 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723832 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723835 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723837 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723840 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723842 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723845 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723847 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723850 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723852 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723855 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723858 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723861 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723864 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723867 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.727315 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723869 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723872 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723875 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723877 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723880 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723882 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723886 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723889 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723892 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723895 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723898 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723900 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723903 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723906 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723908 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723911 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723913 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723916 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723919 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.727828 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723922 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723924 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723926 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723929 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723931 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723934 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723936 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723938 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723941 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723944 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723946 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723949 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723953 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723956 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723959 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723962 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723965 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723968 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723971 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723973 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.728327 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723976 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723979 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723981 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723984 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723986 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723990 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723993 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723995 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.723998 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724001 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724004 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724006 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724009 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724011 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724014 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724016 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724018 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724021 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724024 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724026 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.728856 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724029 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724031 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724034 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724036 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724039 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724041 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724044 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724046 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724048 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724051 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724053 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724056 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724058 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724060 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724063 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724065 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724068 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724070 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724073 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724149 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724155 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:23:38.729349 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724161 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724165 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724169 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724173 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724177 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724181 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724184 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724187 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724190 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724193 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724196 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724199 2555 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724202 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724205 2555 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724208 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724211 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724214 2555 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724217 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724220 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724223 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724226 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724231 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724233 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:23:38.729867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724236 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724239 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724242 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724245 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724248 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724251 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724253 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724257 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724260 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724263 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724266 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724269 2555 flags.go:64] FLAG: --enable-server="true" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724272 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724277 2555 flags.go:64] FLAG: --event-burst="100" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724280 2555 flags.go:64] FLAG: --event-qps="50" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724283 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724285 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724288 2555 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724291 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724294 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724300 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724303 2555 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724306 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724309 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724311 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:23:38.730420 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724314 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724317 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724320 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724323 2555 flags.go:64] FLAG: --feature-gates="" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724326 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724329 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724332 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724335 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724338 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724340 2555 flags.go:64] FLAG: --help="false" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724343 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-133-207.ec2.internal" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724346 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724348 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724351 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724354 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724357 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724360 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724363 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724365 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724368 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724370 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724380 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724383 2555 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724385 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:23:38.731109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724388 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724391 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724393 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724399 2555 flags.go:64] FLAG: --lock-file="" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724401 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724404 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724407 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724412 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724415 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724417 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724420 2555 flags.go:64] FLAG: --logging-format="text" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724423 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724426 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724429 2555 flags.go:64] FLAG: --manifest-url="" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724432 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724436 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724439 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724442 2555 flags.go:64] FLAG: --max-pods="110" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724445 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724448 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724450 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724453 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724456 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724459 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724461 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:23:38.731782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724468 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724470 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724473 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724477 2555 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724481 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724486 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724489 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724492 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724495 2555 flags.go:64] FLAG: --port="10250" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724497 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724502 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b5763dfcc9378266" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724505 2555 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724507 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724510 2555 flags.go:64] FLAG: --register-node="true" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724513 2555 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724516 2555 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724519 2555 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724521 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724524 2555 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724527 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724530 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724533 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724536 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724538 2555 flags.go:64] FLAG: --runonce="false" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724541 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:23:38.732768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724544 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724546 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724549 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724552 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724554 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724557 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724560 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724563 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724565 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724568 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724571 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724573 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724577 2555 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724580 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724585 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724601 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724604 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724610 2555 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724613 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724615 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724618 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724621 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724624 2555 flags.go:64] FLAG: --v="2" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724628 2555 flags.go:64] FLAG: --version="false" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724631 2555 flags.go:64] FLAG: --vmodule="" Jul 06 19:23:38.733827 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724635 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.724638 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724749 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724752 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724755 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724759 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724762 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724764 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724767 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724769 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724772 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724775 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724777 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724780 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724782 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724785 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724787 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724790 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724792 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724795 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.734549 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724798 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724801 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724804 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724806 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724810 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724812 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724815 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724818 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724820 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724822 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724825 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724827 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724830 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724832 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724835 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724837 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724840 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724842 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724845 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724847 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.735534 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724850 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724852 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724855 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724857 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724860 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724862 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724864 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724867 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724869 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724871 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724873 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724877 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724880 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724882 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724886 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724890 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724894 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724896 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724899 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.736318 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724902 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724904 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724907 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724925 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724929 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724932 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724935 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724937 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724940 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724943 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724945 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724948 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724951 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724953 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724956 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724958 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724960 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724963 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724965 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724968 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.736952 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724970 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724973 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724975 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724977 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724981 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724983 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724986 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724990 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724993 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.724997 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725000 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725002 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725005 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725007 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725010 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725013 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725017 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725019 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725022 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.737855 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725024 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725027 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.725029 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.725582 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.734022 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.734042 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.734668 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.734680 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734771 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734779 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734785 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734789 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734794 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734799 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734804 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.738462 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734808 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734814 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734818 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734822 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734826 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734830 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734835 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734839 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734843 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734847 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734851 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734856 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734860 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734864 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734868 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734872 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734877 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734884 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734890 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.738902 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734894 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734899 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734913 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734920 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734925 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734930 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734935 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734939 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734944 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734948 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734952 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734956 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734960 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734964 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734968 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734972 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734977 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734981 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734985 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734991 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.739473 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.734996 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735001 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735006 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735011 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735015 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735019 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735023 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735027 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735032 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735036 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735040 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735044 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735049 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735053 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735057 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735062 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735067 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735071 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735075 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735079 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.740022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735084 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735088 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735092 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735096 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735100 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735104 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735109 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735113 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735117 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735120 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735125 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735129 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735133 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735145 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735150 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735157 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735161 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735165 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735170 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.740552 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735174 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735178 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735182 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735187 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735191 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735195 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735199 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735203 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735207 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735212 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735216 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735221 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735225 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735229 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.735237 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:38.741073 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735436 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735444 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735449 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735453 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735458 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735462 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735467 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735471 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735475 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735479 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735483 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735487 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735503 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735508 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735512 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735519 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735525 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735530 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735534 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735538 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:38.741507 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735542 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735546 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735550 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735554 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735558 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735562 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735567 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735572 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735576 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735580 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735585 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735606 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735611 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735615 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735619 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735623 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735627 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735631 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735635 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735639 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:38.742051 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735643 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735647 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735651 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735657 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735661 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735673 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735678 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735682 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735687 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735691 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735695 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735699 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735703 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735706 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735711 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735715 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735718 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735725 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735731 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:38.742833 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735735 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735740 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735744 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735748 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735752 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735756 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735760 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735764 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735768 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735773 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735777 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735781 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735785 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735789 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735793 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735797 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735801 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735805 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735809 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735823 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:38.743618 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735829 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735834 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735839 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735844 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735848 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735853 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735857 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735860 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735865 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735869 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735873 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735877 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735881 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735885 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735889 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735893 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735897 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735901 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735905 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:38.744469 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:38.735909 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.735917 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.736072 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.739080 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.740375 2555 server.go:1019] "Starting client certificate rotation" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.740487 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.740523 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.742884 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:23:38.745097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.744731 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:23:38.754139 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.754123 2555 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:23:38.756469 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.756456 2555 log.go:25] "Validated CRI v1 image API" Jul 06 19:23:38.757258 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.757239 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:23:38.759141 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.759125 2555 fs.go:135] Filesystem UUIDs: map[40696a64-50a6-4e25-a72b-9c106c7e658e:/dev/nvme0n1p4 720b272d-eb7b-4d1a-95ee-3267f3f51c2b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 06 19:23:38.759189 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.759142 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:23:38.763996 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.763567 2555 manager.go:216] Machine: {Timestamp:2026-07-06 19:23:38.762499935 +0000 UTC m=+0.268098408 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101103 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2adbe1421ce9243437a0ec6965b02a SystemUUID:ec2adbe1-421c-e924-3437-a0ec6965b02a BootID:98510a96-b148-4e36-85ec-0766b340bbe0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:b0:c2:10:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:b0:c2:10:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:f9:07:1b:f2:d9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:23:38.763996 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.763989 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:23:38.764136 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.764100 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:23:38.765346 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.765311 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:23:38.765811 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.765433 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-207.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:23:38.766376 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.766364 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:23:38.766416 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.766380 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:23:38.766416 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.766388 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:23:38.766964 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.766953 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:23:38.768427 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.768417 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:23:38.772125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772113 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:23:38.772234 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772225 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:23:38.772805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772795 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:23:38.772835 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772808 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:23:38.772835 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772821 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:23:38.772835 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772830 2555 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:23:38.772931 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.772838 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:23:38.773251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.773239 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:23:38.773291 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.773260 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:23:38.775054 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.775040 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:23:38.776717 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.776704 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:23:38.776766 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.776735 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:23:38.777815 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777804 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:23:38.777846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777821 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:23:38.777846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777828 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:23:38.777846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777833 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:23:38.777846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777841 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:23:38.777846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777847 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777855 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777860 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777866 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777871 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777879 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:23:38.777976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.777884 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:23:38.778555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.778543 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:23:38.778555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.778554 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:23:38.781937 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.781923 2555 server.go:1268] "Started kubelet" Jul 06 19:23:38.782167 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.782136 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:23:38.782565 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.782532 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-207.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:23:38.782827 ip-10-0-133-207 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:23:38.783006 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.782987 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:23:38.783082 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.782515 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:23:38.783138 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.783108 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:23:38.783275 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.783245 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:23:38.783358 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.783269 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:23:38.784211 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.784198 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:23:38.784312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.784299 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:23:38.786545 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.785806 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-207.ec2.internal.18bfc9c375119100 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-207.ec2.internal,UID:ip-10-0-133-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-207.ec2.internal,},FirstTimestamp:2026-07-06 19:23:38.78189696 +0000 UTC m=+0.287495433,LastTimestamp:2026-07-06 19:23:38.78189696 +0000 UTC m=+0.287495433,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-207.ec2.internal,}" Jul 06 19:23:38.787254 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.787236 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:23:38.790342 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.790325 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:23:38.790417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.790358 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:23:38.790417 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.790408 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:38.791028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.790917 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:23:38.791028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.790920 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:23:38.791028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.790944 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:23:38.791028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791023 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:23:38.791028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791033 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791314 2555 factory.go:153] Registering CRI-O factory Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791329 2555 factory.go:223] Registration of the crio container factory successfully Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791379 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791387 2555 factory.go:55] Registering systemd factory Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791393 2555 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791410 2555 factory.go:103] Registering Raw factory Jul 06 19:23:38.791460 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.791423 2555 manager.go:1196] Started watching for new ooms in manager Jul 06 19:23:38.792100 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.792075 2555 manager.go:349] Starting recovery of all containers Jul 06 19:23:38.792895 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.792873 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:23:38.792986 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.792943 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:23:38.801253 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.801231 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w22fg" Jul 06 19:23:38.804118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.804013 2555 manager.go:354] Recovery completed Jul 06 19:23:38.808159 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.808148 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:38.808507 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.808489 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w22fg" Jul 06 19:23:38.810406 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.810390 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:38.810484 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.810425 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:38.810484 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.810439 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:38.811105 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811088 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:23:38.811105 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811096 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:38.811105 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811101 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:23:38.811272 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811119 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:23:38.811779 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811761 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:38.811861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811787 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:38.811861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.811797 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:38.813046 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.813033 2555 policy_none.go:49] "None policy: Start" Jul 06 19:23:38.813118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.813051 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:23:38.813118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.813064 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:23:38.815056 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.815043 2555 policy_none.go:47] "Start" Jul 06 19:23:38.861139 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861118 2555 manager.go:337] "Starting Device Plugin manager" Jul 06 19:23:38.861227 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.861155 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:23:38.861227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861171 2555 server.go:87] "Starting device plugin registration server" Jul 06 19:23:38.861387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861376 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:23:38.861425 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861391 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:23:38.861513 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861498 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:23:38.861678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861606 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:23:38.861678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.861616 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:23:38.862098 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.862080 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:23:38.862182 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.862128 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:38.959605 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.959562 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:23:38.961801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.960842 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:23:38.961801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.960892 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:23:38.961801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.960921 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:23:38.961801 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.960980 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:23:38.961801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.961546 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:38.962543 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.962511 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:38.962646 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.962568 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:38.962646 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.962627 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:38.962747 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.962703 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-207.ec2.internal" Jul 06 19:23:38.963041 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.962987 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:38.971323 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:38.971281 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-207.ec2.internal" Jul 06 19:23:38.971323 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.971301 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-207.ec2.internal\": node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:38.991574 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:38.991558 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.061863 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.061835 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal"] Jul 06 19:23:39.061940 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.061909 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:39.062775 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.062760 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:39.062843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.062791 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:39.062843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.062802 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:39.065108 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.065096 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:39.065263 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.065250 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.065300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.065278 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:39.066006 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.065991 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:39.066075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.066015 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:39.066075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.066027 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:39.066075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.066047 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:39.066075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.066067 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:39.066215 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.066078 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:39.068312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.068298 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.068378 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.068322 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:39.069048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.069034 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:39.069048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.069058 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:39.069206 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.069072 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:39.092474 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.092454 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.093629 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.093613 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66f1f7c007b6c83ee623a99644ad20e8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-207.ec2.internal\" (UID: \"66f1f7c007b6c83ee623a99644ad20e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.093707 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.093651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.093707 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.093679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.095208 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.095192 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-207.ec2.internal\" not found" node="ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.100885 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.100865 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-207.ec2.internal\" not found" node="ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.193485 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.193453 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.194579 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194563 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66f1f7c007b6c83ee623a99644ad20e8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-207.ec2.internal\" (UID: \"66f1f7c007b6c83ee623a99644ad20e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.194642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.194642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194627 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.194703 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194669 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.194739 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194678 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66f1f7c007b6c83ee623a99644ad20e8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-207.ec2.internal\" (UID: \"66f1f7c007b6c83ee623a99644ad20e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.194739 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.194691 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84e8bbc1e24dd78018030f6419369d36-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal\" (UID: \"84e8bbc1e24dd78018030f6419369d36\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.294145 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.294078 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.394509 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.394482 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.398702 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.398691 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.404555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.404539 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.495549 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.495512 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-207.ec2.internal\" not found" Jul 06 19:23:39.587200 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.587120 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:39.590942 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.590928 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.600992 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.600972 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:23:39.602505 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.602493 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" Jul 06 19:23:39.609810 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.609794 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:23:39.692453 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.692423 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:39.739898 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.739860 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:23:39.740300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.740000 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:23:39.740300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.740030 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:23:39.740300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.740031 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:23:39.773182 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.773158 2555 apiserver.go:52] "Watching apiserver" Jul 06 19:23:39.791130 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.791104 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:23:39.791293 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.791279 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:23:39.791688 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.791666 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nr7ss","openshift-multus/multus-rwrh7","kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wt87r","openshift-multus/network-metrics-daemon-vphct","openshift-network-diagnostics/network-check-target-94d85","openshift-network-operator/iptables-alerter-x8pps","openshift-ovn-kubernetes/ovnkube-node-ph6kd","kube-system/global-pull-secret-syncer-ff8dv","kube-system/konnectivity-agent-9dhl7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p","openshift-dns/node-resolver-rdrrh","openshift-image-registry/node-ca-6dd5x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal"] Jul 06 19:23:39.794268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.794252 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:39.796297 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.796278 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:39.796810 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.796794 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.797040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.797023 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-krkft\"" Jul 06 19:23:39.797040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.797036 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.797166 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.797036 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:23:39.798653 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.798633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.798748 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.798671 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:23:39.798748 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.798743 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.798968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.798952 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hd2z8\"" Jul 06 19:23:39.799090 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.799071 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.800872 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.800859 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.801016 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.801000 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:23:39.801178 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.801164 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:23:39.801333 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.801320 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.801380 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.801365 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.803229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.803202 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:23:39.803698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.803557 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q8jwm\"" Jul 06 19:23:39.803954 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.803933 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:39.804341 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.804315 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:39.804722 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.804687 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:23:39.804862 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.804775 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.804862 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.804849 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6j4f\"" Jul 06 19:23:39.804974 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.804777 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.806024 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.806004 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:23:39.806440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.806415 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:39.806536 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.806515 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:39.809792 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.809776 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:39.809867 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:39.809838 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:39.811358 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.811313 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:18:39 +0000 UTC" deadline="2027-12-21 10:34:46.258456687 +0000 UTC" Jul 06 19:23:39.811426 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.811358 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12783h11m6.447102101s" Jul 06 19:23:39.812431 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.812413 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.814932 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.814913 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.815277 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815257 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.815365 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815323 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.815365 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815267 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:23:39.815476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815382 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:23:39.815533 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815516 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:23:39.815694 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.815671 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nrvwq\"" Jul 06 19:23:39.817308 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.817295 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5b8b9\"" Jul 06 19:23:39.817372 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.817346 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:23:39.817716 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.817699 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:39.820231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.820215 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dh85b\"" Jul 06 19:23:39.820321 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.820303 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.820417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.820402 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:23:39.820479 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.820454 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:23:39.822922 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.822906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:39.823089 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.823070 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:23:39.823149 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.823104 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.823334 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.823321 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s87gk\"" Jul 06 19:23:39.823393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.823381 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.825298 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.825282 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5d7nv\"" Jul 06 19:23:39.825353 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.825274 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:23:39.825387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.825378 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7qnxz" Jul 06 19:23:39.825554 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.825540 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:23:39.830927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.830907 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7qnxz" Jul 06 19:23:39.894472 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.892385 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:23:39.897506 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e0344a1-a744-4dc7-b794-257feb4ebc13-agent-certs\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:39.897620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897513 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-tmp-dir\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:39.897620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897530 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-cnibin\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.897620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-binary-copy\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.897718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.897718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897650 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/013f9b1e-5971-46e2-9525-fe5115caa274-host-slash\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:39.897718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghrw4\" (UniqueName: \"kubernetes.io/projected/8dc81b78-949e-445a-adb7-cbd3e162034f-kube-api-access-ghrw4\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:39.897718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-os-release\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.897830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897740 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.897830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897759 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-os-release\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.897830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:39.897830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897788 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-system-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.897830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897832 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-sys\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897873 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-tuned\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-multus-certs\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvdzp\" (UniqueName: \"kubernetes.io/projected/290981db-d77c-4e21-95cb-f164bfaacd93-kube-api-access-tvdzp\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-slash\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897942 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.897968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.897986 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-systemd-units\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-script-lib\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898021 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8dc81b78-949e-445a-adb7-cbd3e162034f-serviceca\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898034 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-dbus\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898050 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovn-node-metrics-cert\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwrfn\" (UniqueName: \"kubernetes.io/projected/668b4e5a-6f95-4523-954a-5cc8d64801cd-kube-api-access-bwrfn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898105 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8ljr\" (UniqueName: \"kubernetes.io/projected/a7758425-5533-4c11-9109-06bc17098920-kube-api-access-r8ljr\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898137 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysconfig\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-multus-daemon-config\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.898231 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898225 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-kubernetes\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898253 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-host\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-cni-binary-copy\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898316 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-conf-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898343 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-node-log\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898365 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-bin\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898390 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898415 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898442 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rc4b\" (UniqueName: \"kubernetes.io/projected/4a627e50-6129-422b-b65f-a1926646b6b6-kube-api-access-2rc4b\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-conf\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-systemd\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898520 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-etc-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898541 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-log-socket\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898563 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898631 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6r9w\" (UniqueName: \"kubernetes.io/projected/013f9b1e-5971-46e2-9525-fe5115caa274-kube-api-access-c6r9w\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898662 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-run\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.898678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898685 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-lib-modules\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898703 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-tmp\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898728 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbss\" (UniqueName: \"kubernetes.io/projected/4fea7a27-8687-4fb6-90b2-79c856fff4b8-kube-api-access-jjbss\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898741 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e0344a1-a744-4dc7-b794-257feb4ebc13-konnectivity-ca\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-netns\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898769 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898785 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898800 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvtl\" (UniqueName: \"kubernetes.io/projected/22353e76-49ad-4dda-be56-955c247e7cae-kube-api-access-wxvtl\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-cnibin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898888 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-k8s-cni-cncf-io\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898928 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898958 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-kubelet\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.898993 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-etc-kubernetes\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-hosts-file\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899058 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-socket-dir-parent\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-bin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899093 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-system-cni-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899107 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42tk\" (UniqueName: \"kubernetes.io/projected/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-kube-api-access-b42tk\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899135 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-systemd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-ovn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899172 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-netd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899185 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-config\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899216 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/013f9b1e-5971-46e2-9525-fe5115caa274-iptables-alerter-script\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899251 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-kubelet-config\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899275 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-netns\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899295 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-kubelet\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-var-lib-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-env-overrides\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899347 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899373 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-modprobe-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-hostroot\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899424 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.899966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899449 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-device-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:39.900458 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc81b78-949e-445a-adb7-cbd3e162034f-host\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:39.900458 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899483 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-var-lib-kubelet\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:39.900458 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899498 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:39.900458 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.899519 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-multus\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:39.976546 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:39.976514 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f1f7c007b6c83ee623a99644ad20e8.slice/crio-9638491f6c5515369b1d0367024a6b8aacb6e3885650e43f12cc1e77b6f37b76 WatchSource:0}: Error finding container 9638491f6c5515369b1d0367024a6b8aacb6e3885650e43f12cc1e77b6f37b76: Status 404 returned error can't find the container with id 9638491f6c5515369b1d0367024a6b8aacb6e3885650e43f12cc1e77b6f37b76 Jul 06 19:23:39.976970 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:39.976948 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e8bbc1e24dd78018030f6419369d36.slice/crio-ad5b9c3c2a59fca01a14685717f0b84c72285cb3c853338d9306dee80665e234 WatchSource:0}: Error finding container ad5b9c3c2a59fca01a14685717f0b84c72285cb3c853338d9306dee80665e234: Status 404 returned error can't find the container with id ad5b9c3c2a59fca01a14685717f0b84c72285cb3c853338d9306dee80665e234 Jul 06 19:23:39.982922 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:39.982906 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:23:40.000139 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000117 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-systemd-units\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000237 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-script-lib\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000237 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8dc81b78-949e-445a-adb7-cbd3e162034f-serviceca\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.000237 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000192 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-systemd-units\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000237 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000201 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-dbus\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.000450 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovn-node-metrics-cert\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000450 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwrfn\" (UniqueName: \"kubernetes.io/projected/668b4e5a-6f95-4523-954a-5cc8d64801cd-kube-api-access-bwrfn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000450 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000286 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8ljr\" (UniqueName: \"kubernetes.io/projected/a7758425-5533-4c11-9109-06bc17098920-kube-api-access-r8ljr\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.000450 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000331 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-dbus\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysconfig\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-multus-daemon-config\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000508 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-kubernetes\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000534 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-host\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-cni-binary-copy\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-conf-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000656 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-node-log\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.000686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-bin\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000716 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000749 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000747 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8dc81b78-949e-445a-adb7-cbd3e162034f-serviceca\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rc4b\" (UniqueName: \"kubernetes.io/projected/4a627e50-6129-422b-b65f-a1926646b6b6-kube-api-access-2rc4b\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000792 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-script-lib\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-conf\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-systemd\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000882 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-etc-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-log-socket\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6r9w\" (UniqueName: \"kubernetes.io/projected/013f9b1e-5971-46e2-9525-fe5115caa274-kube-api-access-c6r9w\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000976 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001032 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-run\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001040 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-conf-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.001095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001088 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-systemd\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001149 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-etc-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.000985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-run\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001184 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-node-log\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001188 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-conf\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-log-socket\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-lib-modules\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001240 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001257 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-cni-binary-copy\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-tmp\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001291 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-kubernetes\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001316 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbss\" (UniqueName: \"kubernetes.io/projected/4fea7a27-8687-4fb6-90b2-79c856fff4b8-kube-api-access-jjbss\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001342 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001346 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e0344a1-a744-4dc7-b794-257feb4ebc13-konnectivity-ca\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-netns\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001403 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:40.001848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001450 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvtl\" (UniqueName: \"kubernetes.io/projected/22353e76-49ad-4dda-be56-955c247e7cae-kube-api-access-wxvtl\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001473 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-cnibin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-k8s-cni-cncf-io\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001515 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001538 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-kubelet\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001542 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysconfig\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-etc-kubernetes\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001603 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-hosts-file\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001284 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-bin\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001656 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-socket-dir-parent\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001679 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-lib-modules\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-bin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001713 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-host\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001728 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-bin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-system-cni-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42tk\" (UniqueName: \"kubernetes.io/projected/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-kube-api-access-b42tk\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001789 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-systemd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-ovn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.002813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001847 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-netns\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001841 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-netd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-config\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001894 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001921 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e0344a1-a744-4dc7-b794-257feb4ebc13-konnectivity-ca\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/013f9b1e-5971-46e2-9525-fe5115caa274-iptables-alerter-script\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001955 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001972 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-kubelet-config\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002000 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-netns\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-kubelet\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002075 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-var-lib-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-env-overrides\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002131 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002137 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-etc-kubernetes\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-modprobe-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002182 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-hosts-file\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-hostroot\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.003867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002250 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002262 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-socket-dir-parent\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002255 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-device-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002294 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-cnibin\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002301 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-device-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002310 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc81b78-949e-445a-adb7-cbd3e162034f-host\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-var-lib-kubelet\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002303 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-kubelet\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002574 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dc81b78-949e-445a-adb7-cbd3e162034f-host\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002358 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-var-lib-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002360 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afc2758e-8980-4335-8c35-6453b75a191e-kubelet-config\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002379 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-kubelet\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002625 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-var-lib-kubelet\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001815 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-k8s-cni-cncf-io\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002393 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-netns\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002348 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-cni-netd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002415 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-modprobe-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.004794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002425 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-hostroot\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002461 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002463 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-systemd\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002464 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a627e50-6129-422b-b65f-a1926646b6b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002471 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/013f9b1e-5971-46e2-9525-fe5115caa274-iptables-alerter-script\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002407 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-ovn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002727 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.001764 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-system-cni-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002775 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-multus\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e0344a1-a744-4dc7-b794-257feb4ebc13-agent-certs\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-tmp-dir\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002948 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovnkube-config\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.002845 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002829 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/668b4e5a-6f95-4523-954a-5cc8d64801cd-env-overrides\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002816 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-var-lib-cni-multus\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.002966 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-cnibin\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003012 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-cnibin\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005414 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.003026 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:23:40.503005364 +0000 UTC m=+2.008603852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-binary-copy\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003074 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003100 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/013f9b1e-5971-46e2-9525-fe5115caa274-host-slash\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghrw4\" (UniqueName: \"kubernetes.io/projected/8dc81b78-949e-445a-adb7-cbd3e162034f-kube-api-access-ghrw4\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-os-release\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003174 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003199 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-os-release\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003225 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-system-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003266 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-tmp-dir\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003276 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003300 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-run-openvswitch\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003357 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-sys\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003409 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-tuned\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003444 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-multus-certs\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.005953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003446 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003504 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7758425-5533-4c11-9109-06bc17098920-os-release\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003517 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-os-release\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003199 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/013f9b1e-5971-46e2-9525-fe5115caa274-host-slash\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003569 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-sys\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003579 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-binary-copy\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.003606 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003637 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-host-run-multus-certs\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.003652 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:23:40.503635436 +0000 UTC m=+2.009233901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003673 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003681 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvdzp\" (UniqueName: \"kubernetes.io/projected/290981db-d77c-4e21-95cb-f164bfaacd93-kube-api-access-tvdzp\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-slash\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003730 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-system-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003744 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003776 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/668b4e5a-6f95-4523-954a-5cc8d64801cd-host-slash\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003713 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7758425-5533-4c11-9109-06bc17098920-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.006700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003856 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-sysctl-d\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.003894 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/290981db-d77c-4e21-95cb-f164bfaacd93-multus-cni-dir\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.004440 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/290981db-d77c-4e21-95cb-f164bfaacd93-multus-daemon-config\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.005177 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-tmp\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.005291 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/668b4e5a-6f95-4523-954a-5cc8d64801cd-ovn-node-metrics-cert\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.006122 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fea7a27-8687-4fb6-90b2-79c856fff4b8-etc-tuned\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.007241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.006660 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e0344a1-a744-4dc7-b794-257feb4ebc13-agent-certs\") pod \"konnectivity-agent-9dhl7\" (UID: \"1e0344a1-a744-4dc7-b794-257feb4ebc13\") " pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:40.015927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.015907 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rc4b\" (UniqueName: \"kubernetes.io/projected/4a627e50-6129-422b-b65f-a1926646b6b6-kube-api-access-2rc4b\") pod \"aws-ebs-csi-driver-node-pnb5p\" (UID: \"4a627e50-6129-422b-b65f-a1926646b6b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.018590 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.018541 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghrw4\" (UniqueName: \"kubernetes.io/projected/8dc81b78-949e-445a-adb7-cbd3e162034f-kube-api-access-ghrw4\") pod \"node-ca-6dd5x\" (UID: \"8dc81b78-949e-445a-adb7-cbd3e162034f\") " pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.019511 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.018993 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8ljr\" (UniqueName: \"kubernetes.io/projected/a7758425-5533-4c11-9109-06bc17098920-kube-api-access-r8ljr\") pod \"multus-additional-cni-plugins-nr7ss\" (UID: \"a7758425-5533-4c11-9109-06bc17098920\") " pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.019511 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.019127 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6r9w\" (UniqueName: \"kubernetes.io/projected/013f9b1e-5971-46e2-9525-fe5115caa274-kube-api-access-c6r9w\") pod \"iptables-alerter-x8pps\" (UID: \"013f9b1e-5971-46e2-9525-fe5115caa274\") " pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.019511 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.019434 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwrfn\" (UniqueName: \"kubernetes.io/projected/668b4e5a-6f95-4523-954a-5cc8d64801cd-kube-api-access-bwrfn\") pod \"ovnkube-node-ph6kd\" (UID: \"668b4e5a-6f95-4523-954a-5cc8d64801cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.021484 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.020034 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:40.021484 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.020068 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:40.021484 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.020093 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:40.021484 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.020168 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:40.52014174 +0000 UTC m=+2.025740227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:40.021927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.021904 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvdzp\" (UniqueName: \"kubernetes.io/projected/290981db-d77c-4e21-95cb-f164bfaacd93-kube-api-access-tvdzp\") pod \"multus-rwrh7\" (UID: \"290981db-d77c-4e21-95cb-f164bfaacd93\") " pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.022006 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.021906 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbss\" (UniqueName: \"kubernetes.io/projected/4fea7a27-8687-4fb6-90b2-79c856fff4b8-kube-api-access-jjbss\") pod \"tuned-wt87r\" (UID: \"4fea7a27-8687-4fb6-90b2-79c856fff4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.022566 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.022545 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42tk\" (UniqueName: \"kubernetes.io/projected/d8c2649c-87a9-47ab-9ae1-1b68ae762cbe-kube-api-access-b42tk\") pod \"node-resolver-rdrrh\" (UID: \"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe\") " pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.022696 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.022594 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvtl\" (UniqueName: \"kubernetes.io/projected/22353e76-49ad-4dda-be56-955c247e7cae-kube-api-access-wxvtl\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:40.119432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.119359 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8pps" Jul 06 19:23:40.130522 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.130496 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013f9b1e_5971_46e2_9525_fe5115caa274.slice/crio-98416c3d8edc55445feb6c8e843b3468fcb2679906f9ac9cd7a4fe104b2b5221 WatchSource:0}: Error finding container 98416c3d8edc55445feb6c8e843b3468fcb2679906f9ac9cd7a4fe104b2b5221: Status 404 returned error can't find the container with id 98416c3d8edc55445feb6c8e843b3468fcb2679906f9ac9cd7a4fe104b2b5221 Jul 06 19:23:40.138684 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.138665 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dd5x" Jul 06 19:23:40.148338 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.148319 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc81b78_949e_445a_adb7_cbd3e162034f.slice/crio-6693f2a8dbde44345ea68d123082334c4ab71cd17c67c890707c20f7ad920ebe WatchSource:0}: Error finding container 6693f2a8dbde44345ea68d123082334c4ab71cd17c67c890707c20f7ad920ebe: Status 404 returned error can't find the container with id 6693f2a8dbde44345ea68d123082334c4ab71cd17c67c890707c20f7ad920ebe Jul 06 19:23:40.156028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.156012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:23:40.164681 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.164647 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668b4e5a_6f95_4523_954a_5cc8d64801cd.slice/crio-7b3a855a2386d56de9ceecb27c9d6f394a006b2cdb3dfb331bf8bae5d60c45a3 WatchSource:0}: Error finding container 7b3a855a2386d56de9ceecb27c9d6f394a006b2cdb3dfb331bf8bae5d60c45a3: Status 404 returned error can't find the container with id 7b3a855a2386d56de9ceecb27c9d6f394a006b2cdb3dfb331bf8bae5d60c45a3 Jul 06 19:23:40.170845 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.170825 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wt87r" Jul 06 19:23:40.181181 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.181162 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fea7a27_8687_4fb6_90b2_79c856fff4b8.slice/crio-fe30bd74145bb302615198c23606dfe975e4e75b26ece2af5cae3b24887940b0 WatchSource:0}: Error finding container fe30bd74145bb302615198c23606dfe975e4e75b26ece2af5cae3b24887940b0: Status 404 returned error can't find the container with id fe30bd74145bb302615198c23606dfe975e4e75b26ece2af5cae3b24887940b0 Jul 06 19:23:40.185255 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.185237 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" Jul 06 19:23:40.190393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.190378 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwrh7" Jul 06 19:23:40.193515 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.193492 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7758425_5533_4c11_9109_06bc17098920.slice/crio-3fac9f4c8c483f8b35349e463c3665bb46edf5157381afafa3b68db7374a8505 WatchSource:0}: Error finding container 3fac9f4c8c483f8b35349e463c3665bb46edf5157381afafa3b68db7374a8505: Status 404 returned error can't find the container with id 3fac9f4c8c483f8b35349e463c3665bb46edf5157381afafa3b68db7374a8505 Jul 06 19:23:40.199086 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.199066 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290981db_d77c_4e21_95cb_f164bfaacd93.slice/crio-71bb7dc6e9d318188b15f65091c815019ac0e7ed06f15fa8dc43fc69b445ae33 WatchSource:0}: Error finding container 71bb7dc6e9d318188b15f65091c815019ac0e7ed06f15fa8dc43fc69b445ae33: Status 404 returned error can't find the container with id 71bb7dc6e9d318188b15f65091c815019ac0e7ed06f15fa8dc43fc69b445ae33 Jul 06 19:23:40.214441 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.214416 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:23:40.223677 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.223651 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0344a1_a744_4dc7_b794_257feb4ebc13.slice/crio-318b7529d578b1aec7edfc7d1eebbbd3800e259c9e2937cad98a55eefc12991c WatchSource:0}: Error finding container 318b7529d578b1aec7edfc7d1eebbbd3800e259c9e2937cad98a55eefc12991c: Status 404 returned error can't find the container with id 318b7529d578b1aec7edfc7d1eebbbd3800e259c9e2937cad98a55eefc12991c Jul 06 19:23:40.227712 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.227692 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" Jul 06 19:23:40.232312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.232287 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rdrrh" Jul 06 19:23:40.236901 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.236879 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a627e50_6129_422b_b65f_a1926646b6b6.slice/crio-1cd9992558363b6324255213ab5e3ab47eaa4605dde1e5ce93ab51452e0f2f72 WatchSource:0}: Error finding container 1cd9992558363b6324255213ab5e3ab47eaa4605dde1e5ce93ab51452e0f2f72: Status 404 returned error can't find the container with id 1cd9992558363b6324255213ab5e3ab47eaa4605dde1e5ce93ab51452e0f2f72 Jul 06 19:23:40.242257 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:23:40.242228 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c2649c_87a9_47ab_9ae1_1b68ae762cbe.slice/crio-ba21dd72e042fa773f6a6a5523f2788da2a933531c11ffad7b36eca5900a4aeb WatchSource:0}: Error finding container ba21dd72e042fa773f6a6a5523f2788da2a933531c11ffad7b36eca5900a4aeb: Status 404 returned error can't find the container with id ba21dd72e042fa773f6a6a5523f2788da2a933531c11ffad7b36eca5900a4aeb Jul 06 19:23:40.289659 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.289637 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:40.508283 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.508242 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:40.508475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.508301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.508475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.508392 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:40.508475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.508425 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:40.508475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.508461 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:23:41.508440252 +0000 UTC m=+3.014038728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:40.508747 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.508481 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:23:41.508470937 +0000 UTC m=+3.014069402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:40.599477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.599450 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:40.609554 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.609390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:40.609816 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.609694 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:40.609816 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.609719 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:40.609816 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.609732 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:40.609816 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.609791 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:41.609771332 +0000 UTC m=+3.115369800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:40.834536 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.834355 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:18:40 +0000 UTC" deadline="2027-12-11 23:01:53.830440311 +0000 UTC" Jul 06 19:23:40.834536 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.834425 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12555h38m12.996019133s" Jul 06 19:23:40.980378 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.979884 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:40.980378 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:40.980017 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:40.987680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.987611 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8pps" event={"ID":"013f9b1e-5971-46e2-9525-fe5115caa274","Type":"ContainerStarted","Data":"98416c3d8edc55445feb6c8e843b3468fcb2679906f9ac9cd7a4fe104b2b5221"} Jul 06 19:23:40.990685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.990602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" event={"ID":"66f1f7c007b6c83ee623a99644ad20e8","Type":"ContainerStarted","Data":"9638491f6c5515369b1d0367024a6b8aacb6e3885650e43f12cc1e77b6f37b76"} Jul 06 19:23:40.992465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:40.992402 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dhl7" event={"ID":"1e0344a1-a744-4dc7-b794-257feb4ebc13","Type":"ContainerStarted","Data":"318b7529d578b1aec7edfc7d1eebbbd3800e259c9e2937cad98a55eefc12991c"} Jul 06 19:23:41.004486 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.004458 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwrh7" event={"ID":"290981db-d77c-4e21-95cb-f164bfaacd93","Type":"ContainerStarted","Data":"71bb7dc6e9d318188b15f65091c815019ac0e7ed06f15fa8dc43fc69b445ae33"} Jul 06 19:23:41.016135 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.016106 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" event={"ID":"84e8bbc1e24dd78018030f6419369d36","Type":"ContainerStarted","Data":"ad5b9c3c2a59fca01a14685717f0b84c72285cb3c853338d9306dee80665e234"} Jul 06 19:23:41.037157 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.036636 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rdrrh" event={"ID":"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe","Type":"ContainerStarted","Data":"ba21dd72e042fa773f6a6a5523f2788da2a933531c11ffad7b36eca5900a4aeb"} Jul 06 19:23:41.064438 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.064339 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" event={"ID":"4a627e50-6129-422b-b65f-a1926646b6b6","Type":"ContainerStarted","Data":"1cd9992558363b6324255213ab5e3ab47eaa4605dde1e5ce93ab51452e0f2f72"} Jul 06 19:23:41.078127 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.078046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerStarted","Data":"3fac9f4c8c483f8b35349e463c3665bb46edf5157381afafa3b68db7374a8505"} Jul 06 19:23:41.086804 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.086666 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wt87r" event={"ID":"4fea7a27-8687-4fb6-90b2-79c856fff4b8","Type":"ContainerStarted","Data":"fe30bd74145bb302615198c23606dfe975e4e75b26ece2af5cae3b24887940b0"} Jul 06 19:23:41.092253 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.092228 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"7b3a855a2386d56de9ceecb27c9d6f394a006b2cdb3dfb331bf8bae5d60c45a3"} Jul 06 19:23:41.103962 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.103904 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dd5x" event={"ID":"8dc81b78-949e-445a-adb7-cbd3e162034f","Type":"ContainerStarted","Data":"6693f2a8dbde44345ea68d123082334c4ab71cd17c67c890707c20f7ad920ebe"} Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.521225 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.521280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.521472 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.521538 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:23:43.521519398 +0000 UTC m=+5.027117867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.521635 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:41.522433 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.521702 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:23:43.521688479 +0000 UTC m=+5.027286945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:41.622155 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.622112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:41.622330 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.622314 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:41.622389 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.622354 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:41.622389 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.622370 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:41.622503 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.622452 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:43.622430836 +0000 UTC m=+5.128029313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:41.834871 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.834819 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:18:40 +0000 UTC" deadline="2028-03-15 04:31:50.558431452 +0000 UTC" Jul 06 19:23:41.834871 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.834860 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14817h8m8.723575572s" Jul 06 19:23:41.961751 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.961473 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:41.961751 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.961632 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:41.961751 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:41.961708 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:41.962045 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:41.961777 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:42.854900 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:42.854861 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:42.910118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:42.910087 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:42.987852 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:42.987822 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:42.988054 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:42.987972 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:43.541037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:43.540895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:43.541037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:43.540947 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:43.541305 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.541077 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:43.541305 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.541104 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:43.541305 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.541142 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:23:47.541123453 +0000 UTC m=+9.046721915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:43.541305 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.541178 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:23:47.541157014 +0000 UTC m=+9.046755490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:43.642077 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:43.642032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:43.642279 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.642267 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:43.642360 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.642293 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:43.642360 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.642306 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:43.642472 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.642373 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:47.642353592 +0000 UTC m=+9.147952060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:43.962516 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:43.961753 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:43.962516 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.961892 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:43.962516 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:43.962323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:43.962516 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:43.962427 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:44.966311 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:44.965823 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:44.966311 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:44.965952 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:45.961899 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:45.961857 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:45.962097 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:45.962029 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:45.962481 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:45.962461 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:45.962585 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:45.962564 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:46.965832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:46.965800 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:46.966244 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:46.965946 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:47.579985 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:47.579946 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:47.580200 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:47.580039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:47.580200 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.580160 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:47.580200 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.580188 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:47.580371 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.580224 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:23:55.580204133 +0000 UTC m=+17.085802610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:47.580371 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.580269 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:23:55.580239467 +0000 UTC m=+17.085837954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:47.680774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:47.680736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:47.681413 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.680950 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:47.681413 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.680986 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:47.681413 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.681001 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:47.681413 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.681069 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:55.681048099 +0000 UTC m=+17.186646568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:47.961876 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:47.961837 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:47.962049 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.961969 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:47.962415 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:47.962387 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:47.962570 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:47.962499 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:48.966551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:48.966135 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:48.966551 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:48.966252 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:49.962000 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:49.961962 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:49.962184 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:49.962099 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:49.962184 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:49.962149 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:49.962306 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:49.962285 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:50.961833 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:50.961803 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:50.962308 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:50.961914 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:51.961803 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:51.961768 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:51.961966 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:51.961890 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:51.962350 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:51.962211 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:51.962350 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:51.962300 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:52.961142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:52.961109 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:52.961309 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:52.961225 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:53.962225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:53.962033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:53.962667 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:53.962033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:53.962667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:53.962310 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:53.962667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:53.962409 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:54.964428 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:54.964398 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:54.964907 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:54.964516 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:55.645040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:55.645004 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:55.645267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:55.645055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:55.645267 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.645130 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:55.645267 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.645143 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:55.645267 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.645206 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:24:11.645186885 +0000 UTC m=+33.150785346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:55.645267 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.645225 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:24:11.645216328 +0000 UTC m=+33.150814803 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:55.745361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:55.745327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:55.745520 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.745498 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:55.745520 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.745521 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:55.745623 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.745531 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:55.745623 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.745610 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:11.745572536 +0000 UTC m=+33.251171019 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:55.961666 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:55.961636 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:55.961820 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:55.961670 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:55.961820 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.961744 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:55.961925 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:55.961866 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:56.961951 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:56.961919 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:56.962369 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:56.962028 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:57.961367 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:57.961336 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:57.961540 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:57.961343 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:57.961540 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:57.961457 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:57.961645 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:57.961566 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:23:58.962345 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:58.962308 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:23:58.962833 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:58.962422 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:23:59.961251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:59.961214 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:23:59.961436 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:59.961333 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:23:59.961436 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:23:59.961371 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:23:59.961563 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:23:59.961457 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:00.962084 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:00.962049 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:00.962765 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:00.962170 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:01.961213 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:01.961182 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:01.961213 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:01.961203 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:01.961440 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:01.961303 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:01.961659 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:01.961631 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:02.163543 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.163513 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" event={"ID":"66f1f7c007b6c83ee623a99644ad20e8","Type":"ContainerStarted","Data":"2d888e7cf6de186f0bfe5c4cc288f1a01c1d9030c300c95f5396deff802e1924"} Jul 06 19:24:02.165068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.165046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwrh7" event={"ID":"290981db-d77c-4e21-95cb-f164bfaacd93","Type":"ContainerStarted","Data":"fa68799cc239b539d4d7915cbe084ae75e3879715aa1a8dfb82f88bf727219b2"} Jul 06 19:24:02.166334 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.166311 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wt87r" event={"ID":"4fea7a27-8687-4fb6-90b2-79c856fff4b8","Type":"ContainerStarted","Data":"6edd9530f0d9aac133b7ec498a8623b3c153e7b92fbb9f86d048a9aa41daedf2"} Jul 06 19:24:02.168313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.168294 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"e22608be6b1a1cbe13b3374bfdbd28d1d5deff03097421171b9b81ce64dadd97"} Jul 06 19:24:02.168389 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.168347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"46b39a5285810c7ef3bf827689e933e12d2588e240195e2917dcb07b13ab6177"} Jul 06 19:24:02.168389 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.168377 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"b0fad7a4b73fd91ead7e87775662d24b03566d08df3c960f83c536a0ed6a3e40"} Jul 06 19:24:02.180095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.180054 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-207.ec2.internal" podStartSLOduration=22.180041301 podStartE2EDuration="22.180041301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:02.179206018 +0000 UTC m=+23.684804501" watchObservedRunningTime="2026-07-06 19:24:02.180041301 +0000 UTC m=+23.685639782" Jul 06 19:24:02.197174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.197132 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wt87r" podStartSLOduration=1.7898702370000001 podStartE2EDuration="23.19711886s" totalImagesPullingTime="21.407248623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:02.196754283 +0000 UTC m=+23.702352764" watchObservedRunningTime="2026-07-06 19:24:02.19711886 +0000 UTC m=+23.702717338" Jul 06 19:24:02.219301 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.219250 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rwrh7" podStartSLOduration=1.784362596 podStartE2EDuration="23.219233997s" totalImagesPullingTime="21.434871401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:02.21885364 +0000 UTC m=+23.724452116" watchObservedRunningTime="2026-07-06 19:24:02.219233997 +0000 UTC m=+23.724832479" Jul 06 19:24:02.962243 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:02.962067 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:02.962383 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:02.962322 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:03.171686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.171556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dd5x" event={"ID":"8dc81b78-949e-445a-adb7-cbd3e162034f","Type":"ContainerStarted","Data":"beef8267a85c58e8baeb1de1f92ed55348cf05fe340bbc2618406b7737e43082"} Jul 06 19:24:03.173101 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.173075 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dhl7" event={"ID":"1e0344a1-a744-4dc7-b794-257feb4ebc13","Type":"ContainerStarted","Data":"2b1e39b647f4b8b5abe69d493f6372a92d4b1caf7d9b1ab1cbd543d8186bd122"} Jul 06 19:24:03.174486 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.174461 2555 generic.go:356] "Generic (PLEG): container finished" podID="84e8bbc1e24dd78018030f6419369d36" containerID="7a05c34b0f40aa1db87d1581abea63f3ecdeb57ae103386befba7518c1be2cb4" exitCode=0 Jul 06 19:24:03.174575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.174536 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" event={"ID":"84e8bbc1e24dd78018030f6419369d36","Type":"ContainerDied","Data":"7a05c34b0f40aa1db87d1581abea63f3ecdeb57ae103386befba7518c1be2cb4"} Jul 06 19:24:03.177688 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.177660 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rdrrh" event={"ID":"d8c2649c-87a9-47ab-9ae1-1b68ae762cbe","Type":"ContainerStarted","Data":"e2ea6db49e78aff5fb95a896d0ea9ca5c683661b3d2f596e3d6a6943393a9316"} Jul 06 19:24:03.179488 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.179464 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" event={"ID":"4a627e50-6129-422b-b65f-a1926646b6b6","Type":"ContainerStarted","Data":"3f1008c208ac2797331ace20e1ce9b9bded43752eec865a5cafcc7ff2bb3bf31"} Jul 06 19:24:03.180885 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.180865 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="40bad32b5da883b710973503bfe7ab0973f91c85657139ed7735ce1498594bea" exitCode=0 Jul 06 19:24:03.180982 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.180929 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"40bad32b5da883b710973503bfe7ab0973f91c85657139ed7735ce1498594bea"} Jul 06 19:24:03.183770 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.183748 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"244bf677e07780bbc83ab81e7d4802bf53028b8263e34cc7e4b13c9b0a2e8f54"} Jul 06 19:24:03.183884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.183866 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"6bcc56c78d7f4c7fe5143fdd0ce4b34284475be5a231f25bff8903d0eb176f68"} Jul 06 19:24:03.183949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.183925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"7fcb0bdb88bf8c488aebdd2d514b61cb654c79924e890cb4a788c2fa43a37872"} Jul 06 19:24:03.191228 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.191193 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6dd5x" podStartSLOduration=11.009200432 podStartE2EDuration="24.191182423s" totalImagesPullingTime="13.181981991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:03.190905096 +0000 UTC m=+24.696503578" watchObservedRunningTime="2026-07-06 19:24:03.191182423 +0000 UTC m=+24.696780905" Jul 06 19:24:03.208449 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.208404 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9dhl7" podStartSLOduration=2.860951317 podStartE2EDuration="24.208386483s" totalImagesPullingTime="21.347435166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:03.208384558 +0000 UTC m=+24.713983042" watchObservedRunningTime="2026-07-06 19:24:03.208386483 +0000 UTC m=+24.713984967" Jul 06 19:24:03.229976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.229930 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rdrrh" podStartSLOduration=2.88362421 podStartE2EDuration="24.229912993s" totalImagesPullingTime="21.346288783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:03.2298198 +0000 UTC m=+24.735418282" watchObservedRunningTime="2026-07-06 19:24:03.229912993 +0000 UTC m=+24.735511472" Jul 06 19:24:03.282893 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.282866 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.368162 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.368125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.471837 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.471811 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.961937 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.961909 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:03.962184 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:03.961909 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:03.962184 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:03.962017 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:03.962184 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:03.962134 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:04.172644 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.172617 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:24:04.187660 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.187627 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" event={"ID":"4a627e50-6129-422b-b65f-a1926646b6b6","Type":"ContainerStarted","Data":"76aaa3e1f6d575f5d5be559daa6d0898a107625a7bcce7d293419a9256cefe3f"} Jul 06 19:24:04.189248 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.189224 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8pps" event={"ID":"013f9b1e-5971-46e2-9525-fe5115caa274","Type":"ContainerStarted","Data":"5a2b272a4dda0aa779ae65b231207950775c4e382519fd251f4ef2daaa1677c5"} Jul 06 19:24:04.191278 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.191244 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" event={"ID":"84e8bbc1e24dd78018030f6419369d36","Type":"ContainerStarted","Data":"2238a0cbb0ebd433ff8650792fb91aaa3ccb139bcf6dfc1128e32cf4cc79db31"} Jul 06 19:24:04.207476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.207427 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x8pps" podStartSLOduration=3.76670238 podStartE2EDuration="25.20741262s" totalImagesPullingTime="21.44071024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:04.207332813 +0000 UTC m=+25.712931296" watchObservedRunningTime="2026-07-06 19:24:04.20741262 +0000 UTC m=+25.713011103" Jul 06 19:24:04.227686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.227617 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-207.ec2.internal" podStartSLOduration=24.227578478 podStartE2EDuration="24.227578478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:04.227268971 +0000 UTC m=+25.732867456" watchObservedRunningTime="2026-07-06 19:24:04.227578478 +0000 UTC m=+25.733176961" Jul 06 19:24:04.336275 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.336234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:04.399288 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.399257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:04.873290 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.872730 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:24:04.172638682Z","UUID":"02231fc5-0146-43b2-8d36-610dc8113839","Handler":null,"Name":"","Endpoint":""} Jul 06 19:24:04.874528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.874506 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:24:04.874778 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.874547 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:24:04.961608 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:04.961567 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:04.961768 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:04.961711 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:05.201565 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:05.201536 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"14da83cbdb462a1c07131e6937fb5ee260931d410fd3dc0b71a6dab3a73fc593"} Jul 06 19:24:05.961842 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:05.961816 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:05.962003 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:05.961923 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:05.962003 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:05.961951 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:05.962003 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:05.961996 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:06.205949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:06.205912 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" event={"ID":"4a627e50-6129-422b-b65f-a1926646b6b6","Type":"ContainerStarted","Data":"446b06640ead2c7c8e9c245987647d0c3f662f0a7abd8e40fe88148f8368c1d2"} Jul 06 19:24:06.962243 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:06.962208 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:06.962411 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:06.962347 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:07.406240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:07.406165 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:24:07.406830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:07.406802 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:24:07.424088 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:07.424038 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnb5p" podStartSLOduration=4.611435799 podStartE2EDuration="28.424021465s" totalImagesPullingTime="23.812585666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:06.226345083 +0000 UTC m=+27.731943565" watchObservedRunningTime="2026-07-06 19:24:07.424021465 +0000 UTC m=+28.929619947" Jul 06 19:24:07.961929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:07.961906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:07.962011 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:07.961918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:07.962066 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:07.962031 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:07.962112 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:07.962079 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:08.212470 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.212389 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="c58d260f35fde6702e30a0175767b035738d3bec2bd759391f821032da0547cf" exitCode=0 Jul 06 19:24:08.212645 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.212476 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"c58d260f35fde6702e30a0175767b035738d3bec2bd759391f821032da0547cf"} Jul 06 19:24:08.216367 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.216341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" event={"ID":"668b4e5a-6f95-4523-954a-5cc8d64801cd","Type":"ContainerStarted","Data":"459397ff34e28fbadceb3857d2ce1614a51baafc0df68dcff40131c29fb544d7"} Jul 06 19:24:08.216615 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.216580 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:24:08.217116 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.217101 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9dhl7" Jul 06 19:24:08.278663 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.278614 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" podStartSLOduration=7.790369999 podStartE2EDuration="29.27858306s" totalImagesPullingTime="21.488213061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:08.278246411 +0000 UTC m=+29.783844893" watchObservedRunningTime="2026-07-06 19:24:08.27858306 +0000 UTC m=+29.784181541" Jul 06 19:24:08.357149 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.357118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:08.964535 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:08.964034 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:08.964535 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:08.964155 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:09.219232 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.218982 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:09.219454 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.219432 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:09.219567 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.219494 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:09.235025 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.235003 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:09.244062 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.244042 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:09.742790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.742739 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94d85"] Jul 06 19:24:09.742925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.742862 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:09.742983 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:09.742961 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:09.745913 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.745885 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ff8dv"] Jul 06 19:24:09.746024 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.745972 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:09.746102 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:09.746082 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:09.761511 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.761490 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vphct"] Jul 06 19:24:09.761660 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:09.761574 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:09.761718 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:09.761686 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:10.223557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:10.223514 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="dc96d409df268ebdade747202507c07b18e6161b7bc1193fc38881fe0927fa3f" exitCode=0 Jul 06 19:24:10.223962 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:10.223620 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"dc96d409df268ebdade747202507c07b18e6161b7bc1193fc38881fe0927fa3f"} Jul 06 19:24:10.961804 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:10.961772 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:10.962027 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:10.961922 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:10.962027 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:10.962004 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:10.962150 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:10.962101 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:11.301895 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.301867 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:11.373671 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.373639 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:11.534647 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.534624 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:11.659614 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.659576 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:11.659698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.659638 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:11.659742 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.659717 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:11.659772 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.659764 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:11.659807 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.659802 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret podName:afc2758e-8980-4335-8c35-6453b75a191e nodeName:}" failed. No retries permitted until 2026-07-06 19:24:43.659790239 +0000 UTC m=+65.165388699 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret") pod "global-pull-secret-syncer-ff8dv" (UID: "afc2758e-8980-4335-8c35-6453b75a191e") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:11.659846 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.659815 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:24:43.659808928 +0000 UTC m=+65.165407387 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:11.760459 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.760427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:11.760569 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.760550 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:11.760569 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.760566 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:11.760660 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.760575 2555 projected.go:196] Error preparing data for projected volume kube-api-access-snngw for pod openshift-network-diagnostics/network-check-target-94d85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:11.760660 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.760643 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw podName:ef38239d-2d98-43bb-8827-2806792a81e4 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:43.760628165 +0000 UTC m=+65.266226678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-snngw" (UniqueName: "kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw") pod "network-check-target-94d85" (UID: "ef38239d-2d98-43bb-8827-2806792a81e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:11.961950 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:11.961924 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:11.962083 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:11.962053 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:12.230207 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:12.230180 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="3c78345ca5ad53219f31c0e512223a684d4509c164384e7c784dc0b993f1c42e" exitCode=0 Jul 06 19:24:12.230355 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:12.230236 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"3c78345ca5ad53219f31c0e512223a684d4509c164384e7c784dc0b993f1c42e"} Jul 06 19:24:12.961796 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:12.961749 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:12.961796 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:12.961778 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:12.962253 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:12.961906 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:24:12.962253 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:12.962057 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ff8dv" podUID="afc2758e-8980-4335-8c35-6453b75a191e" Jul 06 19:24:13.961672 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:13.961631 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:13.961859 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:13.961774 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94d85" podUID="ef38239d-2d98-43bb-8827-2806792a81e4" Jul 06 19:24:14.305152 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.305121 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-207.ec2.internal" event="NodeReady" Jul 06 19:24:14.305312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.305257 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:24:14.376077 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.376040 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-llznx"] Jul 06 19:24:14.379872 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.379839 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.381427 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.381393 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g5hzq"] Jul 06 19:24:14.382816 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.382788 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:24:14.382929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.382843 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fr69w\"" Jul 06 19:24:14.382929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.382886 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:24:14.385519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.385503 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.388551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.388524 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:24:14.388695 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.388634 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:24:14.388919 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.388807 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-96fg5\"" Jul 06 19:24:14.388919 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.388849 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:24:14.393002 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.392977 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llznx"] Jul 06 19:24:14.396560 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.396536 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5hzq"] Jul 06 19:24:14.479221 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479130 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/85e87373-0d72-46c5-ae18-6c4af955b811-config-volume\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.479221 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479178 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d50d1397-7512-454f-9583-a84e4a7b26d4-cert\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.479221 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85e87373-0d72-46c5-ae18-6c4af955b811-tmp-dir\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.479497 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjvb\" (UniqueName: \"kubernetes.io/projected/85e87373-0d72-46c5-ae18-6c4af955b811-kube-api-access-dkjvb\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.479497 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlxfz\" (UniqueName: \"kubernetes.io/projected/d50d1397-7512-454f-9583-a84e4a7b26d4-kube-api-access-qlxfz\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.479497 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.479372 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.571174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.571141 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:14.580065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/85e87373-0d72-46c5-ae18-6c4af955b811-config-volume\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.580065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d50d1397-7512-454f-9583-a84e4a7b26d4-cert\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.580262 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580093 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85e87373-0d72-46c5-ae18-6c4af955b811-tmp-dir\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.580315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjvb\" (UniqueName: \"kubernetes.io/projected/85e87373-0d72-46c5-ae18-6c4af955b811-kube-api-access-dkjvb\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.580315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlxfz\" (UniqueName: \"kubernetes.io/projected/d50d1397-7512-454f-9583-a84e4a7b26d4-kube-api-access-qlxfz\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.580417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.580487 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:14.580468 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:14.580555 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:14.580537 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:15.080518754 +0000 UTC m=+36.586117234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:14.580748 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580729 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85e87373-0d72-46c5-ae18-6c4af955b811-tmp-dir\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.580925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.580908 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/85e87373-0d72-46c5-ae18-6c4af955b811-config-volume\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.585948 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.585828 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d50d1397-7512-454f-9583-a84e4a7b26d4-cert\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.593288 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.593266 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjvb\" (UniqueName: \"kubernetes.io/projected/85e87373-0d72-46c5-ae18-6c4af955b811-kube-api-access-dkjvb\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:14.593541 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.593520 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlxfz\" (UniqueName: \"kubernetes.io/projected/d50d1397-7512-454f-9583-a84e4a7b26d4-kube-api-access-qlxfz\") pod \"ingress-canary-g5hzq\" (UID: \"d50d1397-7512-454f-9583-a84e4a7b26d4\") " pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.674613 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.674554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:14.700733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.700698 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5hzq" Jul 06 19:24:14.961392 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.961362 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:14.961564 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.961367 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:14.964745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.964687 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:24:14.964745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.964734 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:24:14.965160 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:14.964825 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mm8d\"" Jul 06 19:24:15.083881 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:15.083844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:15.084054 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:15.084019 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:15.084112 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:15.084090 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:16.084068661 +0000 UTC m=+37.589667130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:15.962228 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:15.962194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:15.965614 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:15.965564 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2r6mk\"" Jul 06 19:24:15.965993 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:15.965861 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:24:15.966086 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:15.966069 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:24:16.091757 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:16.091719 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:16.091928 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:16.091883 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:16.091985 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:16.091956 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:18.09193646 +0000 UTC m=+39.597534936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:16.280753 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:16.280723 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5hzq"] Jul 06 19:24:16.283754 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:24:16.283719 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50d1397_7512_454f_9583_a84e4a7b26d4.slice/crio-119aff9face2b3610993f2bbbde09a382d3ab3941ada681f2fc2370416529ef7 WatchSource:0}: Error finding container 119aff9face2b3610993f2bbbde09a382d3ab3941ada681f2fc2370416529ef7: Status 404 returned error can't find the container with id 119aff9face2b3610993f2bbbde09a382d3ab3941ada681f2fc2370416529ef7 Jul 06 19:24:17.246884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:17.246651 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="465d52bc12d0298b01926d0729a6233b5b6165e304c9311531c6be2089e511d6" exitCode=0 Jul 06 19:24:17.247703 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:17.246733 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"465d52bc12d0298b01926d0729a6233b5b6165e304c9311531c6be2089e511d6"} Jul 06 19:24:17.248441 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:17.248420 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5hzq" event={"ID":"d50d1397-7512-454f-9583-a84e4a7b26d4","Type":"ContainerStarted","Data":"119aff9face2b3610993f2bbbde09a382d3ab3941ada681f2fc2370416529ef7"} Jul 06 19:24:18.109565 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.109529 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:18.109763 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:18.109676 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:18.109763 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:18.109732 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:22.109717906 +0000 UTC m=+43.615316366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:18.254977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.254940 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7758425-5533-4c11-9109-06bc17098920" containerID="a9f73718b714cdbbc006fc1ef42d888abae2ed7a72841390b8f9dfcea3ea1bbb" exitCode=0 Jul 06 19:24:18.255388 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.255015 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerDied","Data":"a9f73718b714cdbbc006fc1ef42d888abae2ed7a72841390b8f9dfcea3ea1bbb"} Jul 06 19:24:18.523172 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.523137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.622188 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.622153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.720400 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.720364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.875109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.875078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.937985 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:18.937949 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:19.260861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.260826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" event={"ID":"a7758425-5533-4c11-9109-06bc17098920","Type":"ContainerStarted","Data":"d3d6877b2dbbccdbe861905ad66c0b73e5b23bbea0bc3237f867bce976b2e002"} Jul 06 19:24:19.262176 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.262154 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5hzq" event={"ID":"d50d1397-7512-454f-9583-a84e4a7b26d4","Type":"ContainerStarted","Data":"a6a372eabc84388d0d990b0fc259e4e9f5095533394af8fbcff08cdec7e52843"} Jul 06 19:24:19.287359 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.287321 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nr7ss" podStartSLOduration=7.914470921 podStartE2EDuration="40.287309081s" totalImagesPullingTime="32.37283816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:19.285807448 +0000 UTC m=+40.791405930" watchObservedRunningTime="2026-07-06 19:24:19.287309081 +0000 UTC m=+40.792907562" Jul 06 19:24:19.304528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.304339 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g5hzq" podStartSLOduration=2.745986579 podStartE2EDuration="5.30432749s" totalImagesPullingTime="2.558340911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:19.303858736 +0000 UTC m=+40.809457219" watchObservedRunningTime="2026-07-06 19:24:19.30432749 +0000 UTC m=+40.809925971" Jul 06 19:24:19.408143 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.408076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:19.505114 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:19.505070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:22.140113 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:22.140069 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:22.140475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:22.140185 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:22.140475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:22.140250 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:30.140234387 +0000 UTC m=+51.645832848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:30.192499 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:30.192465 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:30.192979 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:30.192584 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:30.192979 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:30.192672 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:46.192658085 +0000 UTC m=+67.698256544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:41.243154 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:41.243127 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph6kd" Jul 06 19:24:41.377861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:41.377832 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:41.478501 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:41.478472 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:43.676475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.676422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:24:43.676475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.676474 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:43.679534 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.679512 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:24:43.679646 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.679629 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:24:43.687028 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:43.687010 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:24:43.687094 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:43.687082 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:25:47.687060553 +0000 UTC m=+129.192659012 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : secret "metrics-daemon-secret" not found Jul 06 19:24:43.690372 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.690353 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afc2758e-8980-4335-8c35-6453b75a191e-original-pull-secret\") pod \"global-pull-secret-syncer-ff8dv\" (UID: \"afc2758e-8980-4335-8c35-6453b75a191e\") " pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:43.777062 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.777021 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:43.780370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.780354 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:24:43.781354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.781338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ff8dv" Jul 06 19:24:43.790221 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.790196 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:24:43.800073 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.800043 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snngw\" (UniqueName: \"kubernetes.io/projected/ef38239d-2d98-43bb-8827-2806792a81e4-kube-api-access-snngw\") pod \"network-check-target-94d85\" (UID: \"ef38239d-2d98-43bb-8827-2806792a81e4\") " pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:43.876050 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.876025 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2r6mk\"" Jul 06 19:24:43.883339 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.883311 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:43.905303 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:43.905274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ff8dv"] Jul 06 19:24:43.908412 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:24:43.908381 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc2758e_8980_4335_8c35_6453b75a191e.slice/crio-e7fd2bf5228086a15946d063583aa683d91303b11880dac0d234ff5006dd6725 WatchSource:0}: Error finding container e7fd2bf5228086a15946d063583aa683d91303b11880dac0d234ff5006dd6725: Status 404 returned error can't find the container with id e7fd2bf5228086a15946d063583aa683d91303b11880dac0d234ff5006dd6725 Jul 06 19:24:44.004977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:44.004911 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94d85"] Jul 06 19:24:44.007223 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:24:44.007200 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef38239d_2d98_43bb_8827_2806792a81e4.slice/crio-a635c1ae3f46470880a9d8a776cbefd3676f31ae2e53af35d11e0a9693add886 WatchSource:0}: Error finding container a635c1ae3f46470880a9d8a776cbefd3676f31ae2e53af35d11e0a9693add886: Status 404 returned error can't find the container with id a635c1ae3f46470880a9d8a776cbefd3676f31ae2e53af35d11e0a9693add886 Jul 06 19:24:44.323161 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:44.323126 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94d85" event={"ID":"ef38239d-2d98-43bb-8827-2806792a81e4","Type":"ContainerStarted","Data":"a635c1ae3f46470880a9d8a776cbefd3676f31ae2e53af35d11e0a9693add886"} Jul 06 19:24:44.324338 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:44.324310 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ff8dv" event={"ID":"afc2758e-8980-4335-8c35-6453b75a191e","Type":"ContainerStarted","Data":"e7fd2bf5228086a15946d063583aa683d91303b11880dac0d234ff5006dd6725"} Jul 06 19:24:46.194380 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:46.194336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:24:46.194831 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:46.194478 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:46.194831 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:24:46.194542 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:18.194525415 +0000 UTC m=+99.700123876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:24:49.339027 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:49.338986 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94d85" event={"ID":"ef38239d-2d98-43bb-8827-2806792a81e4","Type":"ContainerStarted","Data":"01cdf2a1defe7159ee2bc1e497fa488d4f94d8061f47b337ba3b9a487a844f0f"} Jul 06 19:24:49.339452 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:49.339127 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:24:49.340563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:49.340538 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ff8dv" event={"ID":"afc2758e-8980-4335-8c35-6453b75a191e","Type":"ContainerStarted","Data":"e4ba74d20eb5a720557da0202d53c2270dcb8c6f17aa6cb962f01b86f14b4fbd"} Jul 06 19:24:49.361819 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:49.361669 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-94d85" podStartSLOduration=65.648755633 podStartE2EDuration="1m10.36165615s" totalImagesPullingTime="4.712900517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:49.361503624 +0000 UTC m=+70.867102117" watchObservedRunningTime="2026-07-06 19:24:49.36165615 +0000 UTC m=+70.867254632" Jul 06 19:24:49.388707 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:24:49.388649 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ff8dv" podStartSLOduration=65.565682349 podStartE2EDuration="1m10.388628115s" totalImagesPullingTime="4.822945766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:49.388202924 +0000 UTC m=+70.893801408" watchObservedRunningTime="2026-07-06 19:24:49.388628115 +0000 UTC m=+70.894226598" Jul 06 19:25:08.231553 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:08.231444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:09.324370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:09.324303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:18.203457 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:18.203401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:25:18.204044 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:18.203568 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:18.204044 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:18.203700 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls podName:85e87373-0d72-46c5-ae18-6c4af955b811 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:22.203675726 +0000 UTC m=+163.709274200 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls") pod "dns-default-llznx" (UID: "85e87373-0d72-46c5-ae18-6c4af955b811") : secret "dns-default-metrics-tls" not found Jul 06 19:25:20.346159 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:20.346123 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-94d85" Jul 06 19:25:29.959271 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.959236 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t"] Jul 06 19:25:29.961857 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.961839 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:29.962227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.962207 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74779b8599-gtvkx"] Jul 06 19:25:29.964555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.964539 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:25:29.964700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.964681 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rwlf4\"" Jul 06 19:25:29.964769 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.964752 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:29.964911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.964890 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:25:29.967736 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.967717 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:25:29.967835 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.967721 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:25:29.967958 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.967940 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 19:25:29.968064 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968049 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 19:25:29.968127 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968056 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:25:29.968127 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968075 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 19:25:29.968210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968138 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2lqwg\"" Jul 06 19:25:29.968259 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968236 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 19:25:29.968338 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.968323 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:25:29.974313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.974295 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t"] Jul 06 19:25:29.979508 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:29.979489 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74779b8599-gtvkx"] Jul 06 19:25:30.085905 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.085873 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.086072 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.085923 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.086072 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.085970 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmpgd\" (UniqueName: \"kubernetes.io/projected/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-kube-api-access-pmpgd\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.086072 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.086026 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-default-certificate\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.086072 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.086057 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.086227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.086079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-stats-auth\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.086227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.086101 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg7cv\" (UniqueName: \"kubernetes.io/projected/94cabef5-e90c-44fe-aa25-41672255ad22-kube-api-access-rg7cv\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.086227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.086137 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.125553 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.125511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:30.187118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.187267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187124 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmpgd\" (UniqueName: \"kubernetes.io/projected/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-kube-api-access-pmpgd\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.187267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187145 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-default-certificate\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.187267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.187267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-stats-auth\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.187477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg7cv\" (UniqueName: \"kubernetes.io/projected/94cabef5-e90c-44fe-aa25-41672255ad22-kube-api-access-rg7cv\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.187477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.187477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.187390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.187477 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.187435 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:30.687409648 +0000 UTC m=+112.193008114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:30.187698 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.187494 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:30.187698 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.187565 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:25:30.687546802 +0000 UTC m=+112.193145268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:30.187779 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.187709 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:30.187779 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.187777 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:30.687760169 +0000 UTC m=+112.193358629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:30.188210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.188190 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.189806 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.189783 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-stats-auth\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.189918 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.189835 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-default-certificate\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.196721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.196696 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg7cv\" (UniqueName: \"kubernetes.io/projected/94cabef5-e90c-44fe-aa25-41672255ad22-kube-api-access-rg7cv\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.197005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.196988 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmpgd\" (UniqueName: \"kubernetes.io/projected/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-kube-api-access-pmpgd\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.225707 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.225644 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:30.691371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.691268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:30.691371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.691332 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.691371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.691364 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:30.691667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.691424 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:30.691667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.691491 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:31.691473588 +0000 UTC m=+113.197072052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:30.691667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.691494 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:30.691667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.691507 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:25:31.691501225 +0000 UTC m=+113.197099685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:30.691667 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:30.691545 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:31.691528398 +0000 UTC m=+113.197126866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:30.884933 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.884898 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh"] Jul 06 19:25:30.887925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.887909 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" Jul 06 19:25:30.890736 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.890714 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:30.892102 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.892083 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9fdbz\"" Jul 06 19:25:30.892154 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.892112 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:30.899649 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.899628 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh"] Jul 06 19:25:30.993760 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:30.993730 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74sb6\" (UniqueName: \"kubernetes.io/projected/85994fa4-271e-41ea-a0e3-5048ff7ed130-kube-api-access-74sb6\") pod \"volume-data-source-validator-5d456fdbcd-5j2bh\" (UID: \"85994fa4-271e-41ea-a0e3-5048ff7ed130\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" Jul 06 19:25:31.094533 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.094481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74sb6\" (UniqueName: \"kubernetes.io/projected/85994fa4-271e-41ea-a0e3-5048ff7ed130-kube-api-access-74sb6\") pod \"volume-data-source-validator-5d456fdbcd-5j2bh\" (UID: \"85994fa4-271e-41ea-a0e3-5048ff7ed130\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" Jul 06 19:25:31.103970 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.103939 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74sb6\" (UniqueName: \"kubernetes.io/projected/85994fa4-271e-41ea-a0e3-5048ff7ed130-kube-api-access-74sb6\") pod \"volume-data-source-validator-5d456fdbcd-5j2bh\" (UID: \"85994fa4-271e-41ea-a0e3-5048ff7ed130\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" Jul 06 19:25:31.196942 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.196895 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" Jul 06 19:25:31.317575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.317545 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh"] Jul 06 19:25:31.320697 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:25:31.320673 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85994fa4_271e_41ea_a0e3_5048ff7ed130.slice/crio-d0bc05705826bae1075a72a05ea7db15ac16b372e2d9a50028fa9fe616dd8701 WatchSource:0}: Error finding container d0bc05705826bae1075a72a05ea7db15ac16b372e2d9a50028fa9fe616dd8701: Status 404 returned error can't find the container with id d0bc05705826bae1075a72a05ea7db15ac16b372e2d9a50028fa9fe616dd8701 Jul 06 19:25:31.438147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.438107 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" event={"ID":"85994fa4-271e-41ea-a0e3-5048ff7ed130","Type":"ContainerStarted","Data":"d0bc05705826bae1075a72a05ea7db15ac16b372e2d9a50028fa9fe616dd8701"} Jul 06 19:25:31.699783 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.699752 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:31.699937 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.699801 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:31.699937 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:31.699834 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:31.699937 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:31.699906 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:31.699937 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:31.699919 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:31.700089 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:31.699973 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:25:33.699959348 +0000 UTC m=+115.205557807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:31.700089 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:31.699986 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:33.699980448 +0000 UTC m=+115.205578908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:31.700089 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:31.699997 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:33.699991625 +0000 UTC m=+115.205590085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:32.441558 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:32.441522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" event={"ID":"85994fa4-271e-41ea-a0e3-5048ff7ed130","Type":"ContainerStarted","Data":"64c2d05bd2ff86d0981ab0e886004f0e86c65b6e421f573076cedaec6cb32ae4"} Jul 06 19:25:32.462251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:32.462201 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5j2bh" podStartSLOduration=1.518404076 podStartE2EDuration="2.462183915s" totalImagesPullingTime="943.779839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:32.46140677 +0000 UTC m=+113.967005276" watchObservedRunningTime="2026-07-06 19:25:32.462183915 +0000 UTC m=+113.967782418" Jul 06 19:25:32.564098 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:32.564012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:32.656808 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:32.656772 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:33.714821 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:33.714793 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:33.714846 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:33.714938 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:33.714985 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:25:37.714972089 +0000 UTC m=+119.220570552 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:33.714997 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:37.714991216 +0000 UTC m=+119.220589676 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:33.714979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:33.715039 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:33.715233 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:33.715079 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:37.715066838 +0000 UTC m=+119.220665298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:34.908545 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.908516 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kll4p"] Jul 06 19:25:34.911637 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.911621 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:34.914475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.914453 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:25:34.915904 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.915888 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:25:34.915904 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.915897 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:25:34.916009 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.915892 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hv5fq\"" Jul 06 19:25:34.916009 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.915919 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:25:34.928391 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.928368 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kll4p"] Jul 06 19:25:34.973623 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:34.973579 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:35.024897 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.024863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.025061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.024937 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-crio-socket\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.025061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.024969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-data-volume\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.025061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.024989 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.025061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.025003 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cs4m\" (UniqueName: \"kubernetes.io/projected/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-api-access-6cs4m\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.074121 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.074089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:35.125302 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-crio-socket\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-data-volume\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125342 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125357 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cs4m\" (UniqueName: \"kubernetes.io/projected/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-api-access-6cs4m\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125405 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-crio-socket\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125440 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125419 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.125697 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.125442 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:35.125697 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.125509 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:35.625487897 +0000 UTC m=+117.131086376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : configmap "kube-rbac-proxy" not found Jul 06 19:25:35.125697 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.125538 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:35.125697 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.125622 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:35.625582695 +0000 UTC m=+117.131181155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:35.125828 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.125691 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-data-volume\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.138356 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.138329 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cs4m\" (UniqueName: \"kubernetes.io/projected/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-api-access-6cs4m\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.629476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.629434 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.629673 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:35.629524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:35.629673 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.629607 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:35.629673 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.629672 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:36.629656108 +0000 UTC m=+118.135254567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : configmap "kube-rbac-proxy" not found Jul 06 19:25:35.629788 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.629616 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:35.629788 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:35.629733 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:36.62972183 +0000 UTC m=+118.135320291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:36.637750 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:36.637693 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:36.638212 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:36.637770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:36.638212 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:36.637812 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:36.638212 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:36.637872 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.637856257 +0000 UTC m=+120.143454717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : configmap "kube-rbac-proxy" not found Jul 06 19:25:36.638212 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:36.637940 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:36.638212 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:36.638004 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.637986359 +0000 UTC m=+120.143584827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:36.869268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:36.869239 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rdrrh_d8c2649c-87a9-47ab-9ae1-1b68ae762cbe/dns-node-resolver/0.log" Jul 06 19:25:37.746928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:37.746877 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:37.746973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:37.747036 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:37.747059 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:45.747037004 +0000 UTC m=+127.252635477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:37.747114 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:37.747123 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:37.747166 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:25:45.747153197 +0000 UTC m=+127.252751660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:37.747347 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:37.747181 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:45.747173342 +0000 UTC m=+127.252771802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:38.064161 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.064085 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dd5x_8dc81b78-949e-445a-adb7-cbd3e162034f/node-ca/0.log" Jul 06 19:25:38.654089 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.654046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:38.654275 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.654104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:38.654275 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:38.654181 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:38.654275 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:38.654201 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:38.654275 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:38.654251 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:42.654238366 +0000 UTC m=+124.159836826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:38.654275 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:38.654265 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:42.654259146 +0000 UTC m=+124.159857607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : configmap "kube-rbac-proxy" not found Jul 06 19:25:38.672265 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.672243 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5hzq_d50d1397-7512-454f-9583-a84e4a7b26d4/serve-healthcheck-canary/0.log" Jul 06 19:25:38.966829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.966803 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r"] Jul 06 19:25:38.969861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.969847 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:38.972998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.972976 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wzk82\"" Jul 06 19:25:38.974298 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.974281 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 19:25:38.974354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.974286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:38.974417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.974286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:38.974417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.974304 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 19:25:38.981897 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:38.981878 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r"] Jul 06 19:25:39.057487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.057450 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c197b891-612c-4ba1-9cad-abe34cad683b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.057487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.057481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c197b891-612c-4ba1-9cad-abe34cad683b-config\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.057725 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.057527 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcjcw\" (UniqueName: \"kubernetes.io/projected/c197b891-612c-4ba1-9cad-abe34cad683b-kube-api-access-dcjcw\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.157952 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.157916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c197b891-612c-4ba1-9cad-abe34cad683b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.157952 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.157959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c197b891-612c-4ba1-9cad-abe34cad683b-config\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.158225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.158107 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcjcw\" (UniqueName: \"kubernetes.io/projected/c197b891-612c-4ba1-9cad-abe34cad683b-kube-api-access-dcjcw\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.158585 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.158563 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c197b891-612c-4ba1-9cad-abe34cad683b-config\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.160486 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.160467 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c197b891-612c-4ba1-9cad-abe34cad683b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.168727 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.168702 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcjcw\" (UniqueName: \"kubernetes.io/projected/c197b891-612c-4ba1-9cad-abe34cad683b-kube-api-access-dcjcw\") pod \"service-ca-operator-5d46d9ffd4-d865r\" (UID: \"c197b891-612c-4ba1-9cad-abe34cad683b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.278802 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.278727 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" Jul 06 19:25:39.402475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.402403 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r"] Jul 06 19:25:39.404910 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:25:39.404883 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc197b891_612c_4ba1_9cad_abe34cad683b.slice/crio-26debab8f3f1ecb9b6f76d4b860598e286f46a2eb9408a9901a51d5864006076 WatchSource:0}: Error finding container 26debab8f3f1ecb9b6f76d4b860598e286f46a2eb9408a9901a51d5864006076: Status 404 returned error can't find the container with id 26debab8f3f1ecb9b6f76d4b860598e286f46a2eb9408a9901a51d5864006076 Jul 06 19:25:39.460493 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.460454 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" event={"ID":"c197b891-612c-4ba1-9cad-abe34cad683b","Type":"ContainerStarted","Data":"26debab8f3f1ecb9b6f76d4b860598e286f46a2eb9408a9901a51d5864006076"} Jul 06 19:25:39.523387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.523355 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:39.614259 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.614181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:39.677306 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:39.677273 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.467041 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:41.467005 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" event={"ID":"c197b891-612c-4ba1-9cad-abe34cad683b","Type":"ContainerStarted","Data":"a4914dfed9bb2f809d9471c32a3c2b7ef5f49f889f9976ac84e1c77cad9c349a"} Jul 06 19:25:41.491275 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:41.491229 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" podStartSLOduration=1.548044687 podStartE2EDuration="3.49121515s" totalImagesPullingTime="1.943170463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:41.490242632 +0000 UTC m=+122.995841117" watchObservedRunningTime="2026-07-06 19:25:41.49121515 +0000 UTC m=+122.996813640" Jul 06 19:25:41.770823 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:41.770787 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.871650 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:41.871614 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:42.688126 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:42.688087 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:42.688624 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:42.688223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:42.688624 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:42.688263 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:42.688624 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:42.688328 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:42.688624 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:42.688352 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:50.688328016 +0000 UTC m=+132.193926487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:42.688624 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:42.688383 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:50.688366618 +0000 UTC m=+132.193965082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : configmap "kube-rbac-proxy" not found Jul 06 19:25:42.833944 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:42.833911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:42.904949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:42.904919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.277194 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:43.277148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.377068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:43.377037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.547464 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:43.547375 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.645945 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:43.645909 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.165631 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.165582 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-9csdb"] Jul 06 19:25:45.168572 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.168556 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.171211 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.171193 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 19:25:45.171599 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.171572 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 19:25:45.172603 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.172574 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 19:25:45.172690 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.172609 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tgmxt\"" Jul 06 19:25:45.172875 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.172860 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 19:25:45.177846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.177825 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-9csdb"] Jul 06 19:25:45.215828 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.215786 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.304766 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.304733 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.310779 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.310757 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-cabundle\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.310867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.310788 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjqh\" (UniqueName: \"kubernetes.io/projected/20f7cd57-45e7-4730-9c27-55b956e14dfb-kube-api-access-2kjqh\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.310953 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.310934 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-key\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.412094 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.412059 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-cabundle\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.412243 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.412103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjqh\" (UniqueName: \"kubernetes.io/projected/20f7cd57-45e7-4730-9c27-55b956e14dfb-kube-api-access-2kjqh\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.412243 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.412191 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-key\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.412825 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.412802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-cabundle\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.414689 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.414664 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20f7cd57-45e7-4730-9c27-55b956e14dfb-signing-key\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.422630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.422556 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kjqh\" (UniqueName: \"kubernetes.io/projected/20f7cd57-45e7-4730-9c27-55b956e14dfb-kube-api-access-2kjqh\") pod \"service-ca-758f599c5-9csdb\" (UID: \"20f7cd57-45e7-4730-9c27-55b956e14dfb\") " pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.477083 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.477054 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-9csdb" Jul 06 19:25:45.610192 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.610163 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-9csdb"] Jul 06 19:25:45.613531 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:25:45.613508 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f7cd57_45e7_4730_9c27_55b956e14dfb.slice/crio-0abd18a8a3586746d85734bc029fffbd24ee3327e3e933fb11efd186c6b22fdb WatchSource:0}: Error finding container 0abd18a8a3586746d85734bc029fffbd24ee3327e3e933fb11efd186c6b22fdb: Status 404 returned error can't find the container with id 0abd18a8a3586746d85734bc029fffbd24ee3327e3e933fb11efd186c6b22fdb Jul 06 19:25:45.814990 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.814950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:45.815192 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.815034 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:25:45.815192 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:45.815092 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:25:45.815192 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:45.815157 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:01.81513162 +0000 UTC m=+143.320730085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:45.815192 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:45.815178 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:45.815389 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:45.815210 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:45.815389 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:45.815240 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls podName:2033a09b-cb17-417a-b84c-1e1f7eb5f06f nodeName:}" failed. No retries permitted until 2026-07-06 19:26:01.81522589 +0000 UTC m=+143.320824362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-n2j8t" (UID: "2033a09b-cb17-417a-b84c-1e1f7eb5f06f") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:45.815389 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:45.815283 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs podName:94cabef5-e90c-44fe-aa25-41672255ad22 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:01.815268764 +0000 UTC m=+143.320867239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs") pod "router-default-74779b8599-gtvkx" (UID: "94cabef5-e90c-44fe-aa25-41672255ad22") : secret "router-metrics-certs-default" not found Jul 06 19:25:46.482883 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:46.482848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-9csdb" event={"ID":"20f7cd57-45e7-4730-9c27-55b956e14dfb","Type":"ContainerStarted","Data":"ed51d0fb82c5d08d276bd5b111f41430ffc54bc5857d44ad165429081b380c14"} Jul 06 19:25:46.483329 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:46.482897 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-9csdb" event={"ID":"20f7cd57-45e7-4730-9c27-55b956e14dfb","Type":"ContainerStarted","Data":"0abd18a8a3586746d85734bc029fffbd24ee3327e3e933fb11efd186c6b22fdb"} Jul 06 19:25:46.506036 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:46.505987 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-9csdb" podStartSLOduration=1.505972064 podStartE2EDuration="1.505972064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:46.50518046 +0000 UTC m=+128.010778943" watchObservedRunningTime="2026-07-06 19:25:46.505972064 +0000 UTC m=+128.011570545" Jul 06 19:25:47.108425 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.108391 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.217944 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.217907 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.317361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.317326 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.443879 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.443787 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.733386 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.733330 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:25:47.733829 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:47.733484 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:25:47.733829 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:47.733547 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs podName:22353e76-49ad-4dda-be56-955c247e7cae nodeName:}" failed. No retries permitted until 2026-07-06 19:27:49.733532104 +0000 UTC m=+251.239130565 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs") pod "network-metrics-daemon-vphct" (UID: "22353e76-49ad-4dda-be56-955c247e7cae") : secret "metrics-daemon-secret" not found Jul 06 19:25:47.774203 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:47.774171 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:48.137834 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:48.137721 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:48.963857 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:48.963819 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.159538 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:49.159502 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.365473 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:49.365377 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.573343 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:49.573310 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.768262 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:49.768223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.029624 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.029510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.549757 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.549731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.652083 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.652051 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.759028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.758988 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:50.759188 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.759052 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:50.759188 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:50.759184 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:50.759270 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:25:50.759260 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls podName:7e94f7ba-3a50-4f14-bd09-b4ff52d246c1 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:06.759246901 +0000 UTC m=+148.264845362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kll4p" (UID: "7e94f7ba-3a50-4f14-bd09-b4ff52d246c1") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:50.759539 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.759521 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:25:50.963700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:50.963660 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.173106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:51.173071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.366724 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:51.366622 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.468325 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:51.468291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.766251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:51.766214 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.972824 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:51.972786 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:52.359673 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:52.359645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:52.967126 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:52.967099 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:53.499663 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:53.499636 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:53.651178 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:53.651148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:53.766112 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:53.766024 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:53.957376 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:53.957336 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:54.358311 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:54.358275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:54.919201 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:54.919167 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:55.001988 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:55.001960 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:55.571862 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:55.571831 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:55.778626 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:55.778581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:55.962158 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:55.962128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:56.158370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:56.158337 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:56.602193 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:56.602152 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:56.705074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:56.705044 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:56.792947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:56.792913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:56.880550 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:56.880465 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.067904 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.067875 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.169010 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.168914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.270610 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.270563 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.373020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.372986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.474421 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.474388 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.577705 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.577670 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.652790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.652755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:57.777901 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:57.777815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.277225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.277190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.403346 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.403311 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.507350 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.507318 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.708142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.708114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.810078 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.810047 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.883742 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:25:58.883711 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:01.847424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.847392 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:01.847925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.847446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:26:01.847925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.847502 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:01.848137 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.848116 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94cabef5-e90c-44fe-aa25-41672255ad22-service-ca-bundle\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:01.849924 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.849904 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94cabef5-e90c-44fe-aa25-41672255ad22-metrics-certs\") pod \"router-default-74779b8599-gtvkx\" (UID: \"94cabef5-e90c-44fe-aa25-41672255ad22\") " pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:01.850093 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:01.850071 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2033a09b-cb17-417a-b84c-1e1f7eb5f06f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-n2j8t\" (UID: \"2033a09b-cb17-417a-b84c-1e1f7eb5f06f\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:26:02.074829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.074798 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rwlf4\"" Jul 06 19:26:02.079363 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.079349 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2lqwg\"" Jul 06 19:26:02.082477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.082462 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" Jul 06 19:26:02.087401 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.087386 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:02.222850 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.222820 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t"] Jul 06 19:26:02.225442 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:02.225412 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2033a09b_cb17_417a_b84c_1e1f7eb5f06f.slice/crio-591bdf45b17910c3dad256fbc2f17d1fd77a713adc18817ea0b1ec9a7893e310 WatchSource:0}: Error finding container 591bdf45b17910c3dad256fbc2f17d1fd77a713adc18817ea0b1ec9a7893e310: Status 404 returned error can't find the container with id 591bdf45b17910c3dad256fbc2f17d1fd77a713adc18817ea0b1ec9a7893e310 Jul 06 19:26:02.244281 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.244261 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74779b8599-gtvkx"] Jul 06 19:26:02.246717 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:02.246695 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cabef5_e90c_44fe_aa25_41672255ad22.slice/crio-3f8f74d9ec778a06336dca57bffacc76958447205d60da7f297b53292620a482 WatchSource:0}: Error finding container 3f8f74d9ec778a06336dca57bffacc76958447205d60da7f297b53292620a482: Status 404 returned error can't find the container with id 3f8f74d9ec778a06336dca57bffacc76958447205d60da7f297b53292620a482 Jul 06 19:26:02.532735 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.532697 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" event={"ID":"2033a09b-cb17-417a-b84c-1e1f7eb5f06f","Type":"ContainerStarted","Data":"591bdf45b17910c3dad256fbc2f17d1fd77a713adc18817ea0b1ec9a7893e310"} Jul 06 19:26:02.534153 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.534130 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74779b8599-gtvkx" event={"ID":"94cabef5-e90c-44fe-aa25-41672255ad22","Type":"ContainerStarted","Data":"80532d5046b584346ff333d7d34e36362da4f8efa4b23e5e696a037c40bfda7a"} Jul 06 19:26:02.534283 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.534174 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74779b8599-gtvkx" event={"ID":"94cabef5-e90c-44fe-aa25-41672255ad22","Type":"ContainerStarted","Data":"3f8f74d9ec778a06336dca57bffacc76958447205d60da7f297b53292620a482"} Jul 06 19:26:02.559239 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:02.559195 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74779b8599-gtvkx" podStartSLOduration=33.559181015 podStartE2EDuration="33.559181015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:02.558296438 +0000 UTC m=+144.063894921" watchObservedRunningTime="2026-07-06 19:26:02.559181015 +0000 UTC m=+144.064779475" Jul 06 19:26:03.087607 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:03.087555 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:03.090858 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:03.090834 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:03.538074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:03.538046 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:03.539545 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:03.539529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74779b8599-gtvkx" Jul 06 19:26:04.164329 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.164295 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.258737 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.258708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.360073 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.360039 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.469142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.469117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.541999 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.541906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" event={"ID":"2033a09b-cb17-417a-b84c-1e1f7eb5f06f","Type":"ContainerStarted","Data":"5828c4e4404ef13bbcc5c5ed658013315f743bae0ba07bb732e24864a429f3c3"} Jul 06 19:26:04.571884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.571809 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-n2j8t" podStartSLOduration=33.854583643 podStartE2EDuration="35.571793437s" totalImagesPullingTime="1.717209794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:04.571363604 +0000 UTC m=+146.076962087" watchObservedRunningTime="2026-07-06 19:26:04.571793437 +0000 UTC m=+146.077391918" Jul 06 19:26:04.738097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:04.738057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.789284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:06.789247 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:26:06.791617 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:06.791578 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e94f7ba-3a50-4f14-bd09-b4ff52d246c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kll4p\" (UID: \"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1\") " pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:26:07.022968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:07.022935 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hv5fq\"" Jul 06 19:26:07.031268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:07.031245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kll4p" Jul 06 19:26:07.161235 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:07.161207 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kll4p"] Jul 06 19:26:07.164002 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:07.163973 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e94f7ba_3a50_4f14_bd09_b4ff52d246c1.slice/crio-2687bbbccd50698bf124cf814f7c1d44ed48c795a81ca960921740a44b219363 WatchSource:0}: Error finding container 2687bbbccd50698bf124cf814f7c1d44ed48c795a81ca960921740a44b219363: Status 404 returned error can't find the container with id 2687bbbccd50698bf124cf814f7c1d44ed48c795a81ca960921740a44b219363 Jul 06 19:26:07.555882 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:07.555847 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kll4p" event={"ID":"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1","Type":"ContainerStarted","Data":"2687bbbccd50698bf124cf814f7c1d44ed48c795a81ca960921740a44b219363"} Jul 06 19:26:08.467565 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.467536 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:08.471216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.471192 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.473995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.473949 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ztrbz\"" Jul 06 19:26:08.474162 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.474139 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:26:08.474267 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.474214 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:26:08.474347 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.474327 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:26:08.475314 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.475291 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:26:08.475314 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.475308 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:26:08.475487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.475297 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:26:08.475487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.475344 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:26:08.494876 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.494852 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:08.501954 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.501924 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.502103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.501969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjpwk\" (UniqueName: \"kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.502103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.502059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.502240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.502116 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.502240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.502169 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.502240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.502199 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603409 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603377 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603460 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603716 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603681 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjpwk\" (UniqueName: \"kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603772 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.603822 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.603777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.604282 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.604234 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.604395 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.604309 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.604539 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.604500 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.606290 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.606265 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.606421 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.606403 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.621131 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.621103 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjpwk\" (UniqueName: \"kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk\") pod \"console-577c68df98-9lq4r\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:08.638412 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.638392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:08.739563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.739480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:08.784148 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:08.784115 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:10.297891 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.297855 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:26:10.309380 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.309353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315439 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315574 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315454 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315574 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4f8\" (UniqueName: \"kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315574 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315545 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315633 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315670 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.315801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.315769 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.321054 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.321024 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:26:10.323753 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.323734 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:26:10.416526 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416526 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4f8\" (UniqueName: \"kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416628 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416688 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.416805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.416733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.417351 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.417321 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.417543 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.417523 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.417745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.417690 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.417917 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.417892 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.419296 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.419276 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.419452 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.419431 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.426020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.425997 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4f8\" (UniqueName: \"kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8\") pod \"console-798dc8955f-m59c2\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:10.621761 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:10.621681 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:11.030997 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.030973 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:11.034262 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:11.034227 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbd82ee_6d4f_46d9_9316_72be5cd05b73.slice/crio-7ac87ce8d7ca7d2f1c7ee92fc90585ed8ed1ca3123517341f4c738d276eaff56 WatchSource:0}: Error finding container 7ac87ce8d7ca7d2f1c7ee92fc90585ed8ed1ca3123517341f4c738d276eaff56: Status 404 returned error can't find the container with id 7ac87ce8d7ca7d2f1c7ee92fc90585ed8ed1ca3123517341f4c738d276eaff56 Jul 06 19:26:11.045982 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.045955 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:26:11.048675 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:11.048641 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2be36e_6dfa_4ac3_9b9d_bf067c91bf96.slice/crio-9312c664ca7f7105465e7bbbc3191ebe527513248b457e957832b7dfed9c34f5 WatchSource:0}: Error finding container 9312c664ca7f7105465e7bbbc3191ebe527513248b457e957832b7dfed9c34f5: Status 404 returned error can't find the container with id 9312c664ca7f7105465e7bbbc3191ebe527513248b457e957832b7dfed9c34f5 Jul 06 19:26:11.318639 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.318578 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:11.409841 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.409809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:11.570026 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.569932 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kll4p" event={"ID":"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1","Type":"ContainerStarted","Data":"3a7e29edba43bce503cfd8f8f21911fcc9add83a2227457bb3297037db8c9919"} Jul 06 19:26:11.571258 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.571228 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798dc8955f-m59c2" event={"ID":"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96","Type":"ContainerStarted","Data":"9312c664ca7f7105465e7bbbc3191ebe527513248b457e957832b7dfed9c34f5"} Jul 06 19:26:11.572396 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:11.572375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577c68df98-9lq4r" event={"ID":"1cbd82ee-6d4f-46d9-9316-72be5cd05b73","Type":"ContainerStarted","Data":"7ac87ce8d7ca7d2f1c7ee92fc90585ed8ed1ca3123517341f4c738d276eaff56"} Jul 06 19:26:12.578153 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:12.578117 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kll4p" event={"ID":"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1","Type":"ContainerStarted","Data":"2136a8de8105980a1db007332a38f9a7d0bd613cc9ad430ec94b0b714509b329"} Jul 06 19:26:13.341581 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:13.341539 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.402611 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:13.402567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:14.512995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.512953 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:14.594570 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.594540 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb"] Jul 06 19:26:14.600474 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.600447 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.603775 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.603754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:14.605008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.604842 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wk9bf\"" Jul 06 19:26:14.605008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.604893 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 19:26:14.605008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.604913 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 19:26:14.605008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.604843 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:26:14.608167 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.608136 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb"] Jul 06 19:26:14.666441 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.666403 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzsj9\" (UniqueName: \"kubernetes.io/projected/07bdcef8-1b82-4acb-a045-2d540409f00a-kube-api-access-wzsj9\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.666652 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.666462 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdcef8-1b82-4acb-a045-2d540409f00a-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.666652 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.666518 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.666652 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.666585 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.673394 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.673369 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:14.767682 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.767583 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzsj9\" (UniqueName: \"kubernetes.io/projected/07bdcef8-1b82-4acb-a045-2d540409f00a-kube-api-access-wzsj9\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.767682 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.767657 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdcef8-1b82-4acb-a045-2d540409f00a-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.767894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.767688 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.767894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.767746 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.768498 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.768425 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdcef8-1b82-4acb-a045-2d540409f00a-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.770628 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.770578 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.770743 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.770656 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/07bdcef8-1b82-4acb-a045-2d540409f00a-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.778085 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.778060 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzsj9\" (UniqueName: \"kubernetes.io/projected/07bdcef8-1b82-4acb-a045-2d540409f00a-kube-api-access-wzsj9\") pod \"prometheus-operator-567f7b8b7c-vmqlb\" (UID: \"07bdcef8-1b82-4acb-a045-2d540409f00a\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:14.911575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:14.911541 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" Jul 06 19:26:15.328423 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.328377 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:15.430394 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.430363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:15.519021 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.518990 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb"] Jul 06 19:26:15.521967 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:15.521941 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bdcef8_1b82_4acb_a045_2d540409f00a.slice/crio-f2b1c0ed9c6d122fe6b4fb0d64c1cf97bfa2f9b60bad01867c73392e5973f22b WatchSource:0}: Error finding container f2b1c0ed9c6d122fe6b4fb0d64c1cf97bfa2f9b60bad01867c73392e5973f22b: Status 404 returned error can't find the container with id f2b1c0ed9c6d122fe6b4fb0d64c1cf97bfa2f9b60bad01867c73392e5973f22b Jul 06 19:26:15.525463 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.525429 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:15.590739 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.590650 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577c68df98-9lq4r" event={"ID":"1cbd82ee-6d4f-46d9-9316-72be5cd05b73","Type":"ContainerStarted","Data":"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6"} Jul 06 19:26:15.592583 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.592562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kll4p" event={"ID":"7e94f7ba-3a50-4f14-bd09-b4ff52d246c1","Type":"ContainerStarted","Data":"f84458fa9ae2eefbae4184dec09f207a6b26b907527da22d66d0cb869d962222"} Jul 06 19:26:15.593967 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.593950 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798dc8955f-m59c2" event={"ID":"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96","Type":"ContainerStarted","Data":"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99"} Jul 06 19:26:15.594947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.594929 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" event={"ID":"07bdcef8-1b82-4acb-a045-2d540409f00a","Type":"ContainerStarted","Data":"f2b1c0ed9c6d122fe6b4fb0d64c1cf97bfa2f9b60bad01867c73392e5973f22b"} Jul 06 19:26:15.622679 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.622636 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-577c68df98-9lq4r" podStartSLOduration=3.274314376 podStartE2EDuration="7.622622986s" totalImagesPullingTime="4.34830861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:15.622410203 +0000 UTC m=+157.128008686" watchObservedRunningTime="2026-07-06 19:26:15.622622986 +0000 UTC m=+157.128221465" Jul 06 19:26:15.659248 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.659203 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-798dc8955f-m59c2" podStartSLOduration=1.332735906 podStartE2EDuration="5.659189378s" totalImagesPullingTime="4.326453472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:15.65842057 +0000 UTC m=+157.164019066" watchObservedRunningTime="2026-07-06 19:26:15.659189378 +0000 UTC m=+157.164787860" Jul 06 19:26:15.681094 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.681047 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kll4p" podStartSLOduration=33.618221201 podStartE2EDuration="41.681033318s" totalImagesPullingTime="8.062812117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:25:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:15.679939823 +0000 UTC m=+157.185538304" watchObservedRunningTime="2026-07-06 19:26:15.681033318 +0000 UTC m=+157.186631799" Jul 06 19:26:15.861127 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.861005 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:15.950855 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:15.950802 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.028701 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.028619 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.394644 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:17.394524 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-llznx" podUID="85e87373-0d72-46c5-ae18-6c4af955b811" Jul 06 19:26:17.533857 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.533819 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.599206 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.599179 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.602925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.602901 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llznx" Jul 06 19:26:17.603067 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.602922 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" event={"ID":"07bdcef8-1b82-4acb-a045-2d540409f00a","Type":"ContainerStarted","Data":"d15fa4e07daca49a7c28958cbbacd3a93bfae27fbbd96e5a7fb9aaa59890cd73"} Jul 06 19:26:17.603067 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.602976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" event={"ID":"07bdcef8-1b82-4acb-a045-2d540409f00a","Type":"ContainerStarted","Data":"0e3fbde99ff5a5147108adf6723f7c1838a79ed27b38c50b248ece9e2bcad4db"} Jul 06 19:26:17.627998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:17.627951 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vmqlb" podStartSLOduration=2.444117215 podStartE2EDuration="3.62793494s" totalImagesPullingTime="1.183817725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:17.627107275 +0000 UTC m=+159.132705757" watchObservedRunningTime="2026-07-06 19:26:17.62793494 +0000 UTC m=+159.133533421" Jul 06 19:26:17.977994 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:17.977944 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vphct" podUID="22353e76-49ad-4dda-be56-955c247e7cae" Jul 06 19:26:18.784987 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:18.784953 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:18.785368 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:18.785009 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:18.789796 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:18.789778 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:19.614612 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:19.614514 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:19.760147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:19.760118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:19.889228 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:19.889158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:19.995834 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:19.995801 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.178378 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.178286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.289465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.289429 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gwfnc"] Jul 06 19:26:20.293751 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.293730 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.293935 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.293911 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p"] Jul 06 19:26:20.296885 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.296864 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:26:20.297097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.297064 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:26:20.297097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.297069 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sgsxh\"" Jul 06 19:26:20.297406 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.297393 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.298181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.298168 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:26:20.301476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.301458 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 19:26:20.302052 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.302036 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 19:26:20.302166 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.302090 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ddbz6\"" Jul 06 19:26:20.302266 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.302097 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:26:20.317262 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.317240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.319537 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.319518 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p"] Jul 06 19:26:20.416945 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.416915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6befcbd0-7eb2-498e-8d6f-df9177fee622-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.416945 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.416947 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.416972 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417024 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417054 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417109 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417129 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417147 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-wtmp\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-textfile\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417243 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-metrics-client-ca\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417266 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nft9k\" (UniqueName: \"kubernetes.io/projected/22026970-8d60-42d1-9332-6cf85027cd95-kube-api-access-nft9k\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417298 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxrzl\" (UniqueName: \"kubernetes.io/projected/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-api-access-hxrzl\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417314 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-sys\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.417519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.417364 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-root\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.518734 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518742 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-wtmp\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.518894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.518893 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-textfile\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519180 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:20.518964 2555 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 19:26:20.519180 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:20.519043 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls podName:6befcbd0-7eb2-498e-8d6f-df9177fee622 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:21.019022228 +0000 UTC m=+162.524620703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-86k7p" (UID: "6befcbd0-7eb2-498e-8d6f-df9177fee622") : secret "kube-state-metrics-tls" not found Jul 06 19:26:20.519304 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519280 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-textfile\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519402 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:20.519387 2555 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 19:26:20.519465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519417 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-wtmp\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-metrics-client-ca\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519465 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:20.519449 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls podName:22026970-8d60-42d1-9332-6cf85027cd95 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:21.01943267 +0000 UTC m=+162.525031135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls") pod "node-exporter-gwfnc" (UID: "22026970-8d60-42d1-9332-6cf85027cd95") : secret "node-exporter-tls" not found Jul 06 19:26:20.519634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519480 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519493 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nft9k\" (UniqueName: \"kubernetes.io/projected/22026970-8d60-42d1-9332-6cf85027cd95-kube-api-access-nft9k\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxrzl\" (UniqueName: \"kubernetes.io/projected/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-api-access-hxrzl\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.519634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-sys\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519627 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-root\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519668 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6befcbd0-7eb2-498e-8d6f-df9177fee622-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.519884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519699 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.519884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519727 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.519884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519786 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-root\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.519884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.519837 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22026970-8d60-42d1-9332-6cf85027cd95-metrics-client-ca\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.520112 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.520035 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22026970-8d60-42d1-9332-6cf85027cd95-sys\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.520168 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.520133 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6befcbd0-7eb2-498e-8d6f-df9177fee622-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.520364 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.520345 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.520494 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.520476 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6befcbd0-7eb2-498e-8d6f-df9177fee622-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.521538 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.521518 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.522116 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.522097 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.532024 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.531997 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxrzl\" (UniqueName: \"kubernetes.io/projected/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-api-access-hxrzl\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:20.532342 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.532321 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nft9k\" (UniqueName: \"kubernetes.io/projected/22026970-8d60-42d1-9332-6cf85027cd95-kube-api-access-nft9k\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:20.591238 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.591208 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.622680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.622653 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:20.622819 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.622726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:20.627378 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.627358 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:20.688095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.688066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.796334 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.796245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.897552 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.897512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.989195 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:20.989170 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.024152 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.024124 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:21.024299 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.024166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:21.026638 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.026583 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22026970-8d60-42d1-9332-6cf85027cd95-node-exporter-tls\") pod \"node-exporter-gwfnc\" (UID: \"22026970-8d60-42d1-9332-6cf85027cd95\") " pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:21.026638 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.026626 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6befcbd0-7eb2-498e-8d6f-df9177fee622-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-86k7p\" (UID: \"6befcbd0-7eb2-498e-8d6f-df9177fee622\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:21.090090 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.090010 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.193616 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.193562 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:26:21.197900 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.197875 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.198508 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.198486 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.201244 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201221 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:26:21.201339 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201288 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:26:21.201339 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201301 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:26:21.201339 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201289 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ddwmh\"" Jul 06 19:26:21.201689 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201671 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:26:21.201796 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201724 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:26:21.201863 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.201800 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:26:21.202503 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.202486 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:26:21.202633 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.202510 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:26:21.205560 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.205531 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwfnc" Jul 06 19:26:21.207708 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.207691 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:26:21.211433 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.211417 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" Jul 06 19:26:21.215769 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:21.215747 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22026970_8d60_42d1_9332_6cf85027cd95.slice/crio-f026f904ec13e156c9911e73a61ccc062ef7205b6a3a338f08d9e730662a6ff4 WatchSource:0}: Error finding container f026f904ec13e156c9911e73a61ccc062ef7205b6a3a338f08d9e730662a6ff4: Status 404 returned error can't find the container with id f026f904ec13e156c9911e73a61ccc062ef7205b6a3a338f08d9e730662a6ff4 Jul 06 19:26:21.227842 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.227761 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:26:21.327097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327152 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327184 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327206 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-web-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327225 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327388 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327317 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327388 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327347 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327388 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327364 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327393 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327690 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327548 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msxqd\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-kube-api-access-msxqd\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.327690 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.327578 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-out\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.348628 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.348539 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p"] Jul 06 19:26:21.353616 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:21.353574 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6befcbd0_7eb2_498e_8d6f_df9177fee622.slice/crio-13bc9505f9b3f7fa811c10aaa22849eb70ec9bdca6a89a92a67f6d3cfc1102d9 WatchSource:0}: Error finding container 13bc9505f9b3f7fa811c10aaa22849eb70ec9bdca6a89a92a67f6d3cfc1102d9: Status 404 returned error can't find the container with id 13bc9505f9b3f7fa811c10aaa22849eb70ec9bdca6a89a92a67f6d3cfc1102d9 Jul 06 19:26:21.360040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.360013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.428730 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428680 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428730 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428759 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428813 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msxqd\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-kube-api-access-msxqd\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428949 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-out\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.428986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.428984 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.429331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.429019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.429331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.429045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.429331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.429067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-web-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.429331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.429094 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.430046 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.430020 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.432823 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.430204 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.432823 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.431329 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.432823 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.432087 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.432823 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.432655 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-web-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.433829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.433806 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.433928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.433817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.433980 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.433952 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.434040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.433994 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.434119 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.434095 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-config-out\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.434215 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.434125 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.435035 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.435015 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.442326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.442301 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msxqd\" (UniqueName: \"kubernetes.io/projected/026bde0f-1fa4-4925-9ac4-c1955a16c0c7-kube-api-access-msxqd\") pod \"alertmanager-main-0\" (UID: \"026bde0f-1fa4-4925-9ac4-c1955a16c0c7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.442787 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.442766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.510037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.509997 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:26:21.618732 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.618633 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" event={"ID":"6befcbd0-7eb2-498e-8d6f-df9177fee622","Type":"ContainerStarted","Data":"13bc9505f9b3f7fa811c10aaa22849eb70ec9bdca6a89a92a67f6d3cfc1102d9"} Jul 06 19:26:21.619896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.619861 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwfnc" event={"ID":"22026970-8d60-42d1-9332-6cf85027cd95","Type":"ContainerStarted","Data":"f026f904ec13e156c9911e73a61ccc062ef7205b6a3a338f08d9e730662a6ff4"} Jul 06 19:26:21.624765 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.624738 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:26:21.665729 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.665696 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:26:21.669317 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:21.669284 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026bde0f_1fa4_4925_9ac4_c1955a16c0c7.slice/crio-4471029de76927d5beab8a5e53777185ac91912c214ee8ab2089682173f12a6f WatchSource:0}: Error finding container 4471029de76927d5beab8a5e53777185ac91912c214ee8ab2089682173f12a6f: Status 404 returned error can't find the container with id 4471029de76927d5beab8a5e53777185ac91912c214ee8ab2089682173f12a6f Jul 06 19:26:21.692237 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.692207 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:21.857891 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.857802 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.956985 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:21.956949 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:22.236808 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.236780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:26:22.239307 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.239231 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/85e87373-0d72-46c5-ae18-6c4af955b811-metrics-tls\") pod \"dns-default-llznx\" (UID: \"85e87373-0d72-46c5-ae18-6c4af955b811\") " pod="openshift-dns/dns-default-llznx" Jul 06 19:26:22.406279 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.406247 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fr69w\"" Jul 06 19:26:22.414661 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.414586 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llznx" Jul 06 19:26:22.634362 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.634332 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"4471029de76927d5beab8a5e53777185ac91912c214ee8ab2089682173f12a6f"} Jul 06 19:26:22.985850 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:22.985807 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llznx"] Jul 06 19:26:22.992036 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:22.992016 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e87373_0d72_46c5_ae18_6c4af955b811.slice/crio-82c4ffaa4863295651aa9291ed66ea2207b471acd6fb852ba5f5e3ca3afd4b0f WatchSource:0}: Error finding container 82c4ffaa4863295651aa9291ed66ea2207b471acd6fb852ba5f5e3ca3afd4b0f: Status 404 returned error can't find the container with id 82c4ffaa4863295651aa9291ed66ea2207b471acd6fb852ba5f5e3ca3afd4b0f Jul 06 19:26:23.203911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.203827 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c7c865454-hxnnh"] Jul 06 19:26:23.207543 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.207523 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.210354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210315 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 19:26:23.210452 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210421 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 19:26:23.210527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210466 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 19:26:23.210527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210501 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8nwr5\"" Jul 06 19:26:23.210705 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210689 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-326l1m4d1meo6\"" Jul 06 19:26:23.210776 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210762 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 19:26:23.210809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.210766 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 19:26:23.225661 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.225637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c7c865454-hxnnh"] Jul 06 19:26:23.276899 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.276868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:23.346848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.346819 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.346959 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.346852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.346959 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.346871 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.347048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.346953 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-grpc-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.347048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.347009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.347048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.347033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gbqw\" (UniqueName: \"kubernetes.io/projected/c381d033-1248-471e-b89b-edfd4a7c490b-kube-api-access-7gbqw\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.347136 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.347067 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.347136 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.347116 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c381d033-1248-471e-b89b-edfd4a7c490b-metrics-client-ca\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.447885 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.447853 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.447889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gbqw\" (UniqueName: \"kubernetes.io/projected/c381d033-1248-471e-b89b-edfd4a7c490b-kube-api-access-7gbqw\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.447989 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448168 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.448036 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c381d033-1248-471e-b89b-edfd4a7c490b-metrics-client-ca\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448263 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.448243 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.448288 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448356 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.448313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.448394 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.448373 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-grpc-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.449351 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.449320 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c381d033-1248-471e-b89b-edfd4a7c490b-metrics-client-ca\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.451763 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.451545 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.451763 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.451583 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.451923 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.451872 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-grpc-tls\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.452148 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.452121 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.455382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.452428 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.455382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.454621 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c381d033-1248-471e-b89b-edfd4a7c490b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.462298 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.462276 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gbqw\" (UniqueName: \"kubernetes.io/projected/c381d033-1248-471e-b89b-edfd4a7c490b-kube-api-access-7gbqw\") pod \"thanos-querier-7c7c865454-hxnnh\" (UID: \"c381d033-1248-471e-b89b-edfd4a7c490b\") " pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.513203 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.513166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:23.517079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.517058 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:23.591173 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.591077 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:23.645337 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.645300 2555 generic.go:356] "Generic (PLEG): container finished" podID="026bde0f-1fa4-4925-9ac4-c1955a16c0c7" containerID="9bffd208f64159d8b3a8f38c1905099ef1c79e39502a7757f2717385d477dcab" exitCode=0 Jul 06 19:26:23.645778 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.645657 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerDied","Data":"9bffd208f64159d8b3a8f38c1905099ef1c79e39502a7757f2717385d477dcab"} Jul 06 19:26:23.647569 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.647491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llznx" event={"ID":"85e87373-0d72-46c5-ae18-6c4af955b811","Type":"ContainerStarted","Data":"82c4ffaa4863295651aa9291ed66ea2207b471acd6fb852ba5f5e3ca3afd4b0f"} Jul 06 19:26:23.651021 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.650995 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" event={"ID":"6befcbd0-7eb2-498e-8d6f-df9177fee622","Type":"ContainerStarted","Data":"1cad4a104d0c6aabdfef9dce9d717441494cbf0b5848fa8f5b61c537bb41b233"} Jul 06 19:26:23.651102 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.651050 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" event={"ID":"6befcbd0-7eb2-498e-8d6f-df9177fee622","Type":"ContainerStarted","Data":"3a9d3d94344beacf3d8b31dfadcaad9607c55ec36d2625ad21855234723a5f95"} Jul 06 19:26:23.651102 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.651078 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" event={"ID":"6befcbd0-7eb2-498e-8d6f-df9177fee622","Type":"ContainerStarted","Data":"65fe69917f07ac497b1bc8035113d1d6175b65203973f27f9f0ba66f9623f696"} Jul 06 19:26:23.654576 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.654066 2555 generic.go:356] "Generic (PLEG): container finished" podID="22026970-8d60-42d1-9332-6cf85027cd95" containerID="3d17ca5da229b5823570653ae4be487c56b0c5db52855110f99053836c46f57a" exitCode=0 Jul 06 19:26:23.654576 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.654178 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwfnc" event={"ID":"22026970-8d60-42d1-9332-6cf85027cd95","Type":"ContainerDied","Data":"3d17ca5da229b5823570653ae4be487c56b0c5db52855110f99053836c46f57a"} Jul 06 19:26:23.685321 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.685296 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c7c865454-hxnnh"] Jul 06 19:26:23.772685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.772580 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-86k7p" podStartSLOduration=2.519379782 podStartE2EDuration="3.772558958s" totalImagesPullingTime="1.253179176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:23.771123132 +0000 UTC m=+165.276721617" watchObservedRunningTime="2026-07-06 19:26:23.772558958 +0000 UTC m=+165.278157441" Jul 06 19:26:23.841058 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:23.840994 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:24.019424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.019323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:24.663371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.663285 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwfnc" event={"ID":"22026970-8d60-42d1-9332-6cf85027cd95","Type":"ContainerStarted","Data":"b4fb18a5fee2b8ea523273168e94d2125dbabd833405deb2487e40771db21aca"} Jul 06 19:26:24.663371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.663349 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwfnc" event={"ID":"22026970-8d60-42d1-9332-6cf85027cd95","Type":"ContainerStarted","Data":"b1596bf3da63f2e244871b4f39e6a15602cc0737b38a7e2d4fe95d8d5a551e83"} Jul 06 19:26:24.665107 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.665019 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"36dcc4923a9ed9786f4d7e88923544750d4916ffee73042899eb9b7f070f6f0b"} Jul 06 19:26:24.699116 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.699061 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gwfnc" podStartSLOduration=3.26975505 podStartE2EDuration="4.699045074s" totalImagesPullingTime="1.338547803s" totalInitContainerRuntime="90.742221ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:24.697261429 +0000 UTC m=+166.202859934" watchObservedRunningTime="2026-07-06 19:26:24.699045074 +0000 UTC m=+166.204643556" Jul 06 19:26:24.773102 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.773072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:24.773255 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.773232 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:26:24.777807 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.777783 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.794617 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.794535 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:26:24.810390 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.809026 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b"] Jul 06 19:26:24.812698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.812675 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:24.815718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.815478 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:26:24.815804 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.815774 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mnn4t\"" Jul 06 19:26:24.828471 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.828397 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b"] Jul 06 19:26:24.858633 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.858583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:24.864874 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.864844 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gbpj\" (UniqueName: \"kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.864995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.864893 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.864995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.864938 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.864995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.864965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.864995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.864987 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.865240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.865068 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.865240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.865153 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.965986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n885n\" (UniqueName: \"kubernetes.io/projected/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-kube-api-access-n885n\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:24.966424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:24.966424 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966358 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gbpj\" (UniqueName: \"kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966876 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966825 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.966876 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.966869 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.967228 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.967193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.967997 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.967912 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.969958 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.969106 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.969958 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.969282 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:24.978828 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:24.978805 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gbpj\" (UniqueName: \"kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj\") pod \"console-744bd578-d7pr7\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:25.018910 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.018874 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:25.067439 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.067380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n885n\" (UniqueName: \"kubernetes.io/projected/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-kube-api-access-n885n\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:25.067439 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.067430 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:25.067975 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:25.067615 2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 19:26:25.067975 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:25.067688 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert podName:f9e7d964-7d87-4b1f-83e3-e0a9f05fe156 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:25.567666519 +0000 UTC m=+167.073264987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-7vt7b" (UID: "f9e7d964-7d87-4b1f-83e3-e0a9f05fe156") : secret "monitoring-plugin-cert" not found Jul 06 19:26:25.087937 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.087902 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n885n\" (UniqueName: \"kubernetes.io/projected/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-kube-api-access-n885n\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:25.093730 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.093696 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:25.122856 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.122813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:25.572813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.572772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:25.573004 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:25.572936 2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 19:26:25.573067 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:25.573029 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert podName:f9e7d964-7d87-4b1f-83e3-e0a9f05fe156 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:26.573009336 +0000 UTC m=+168.078607800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-7vt7b" (UID: "f9e7d964-7d87-4b1f-83e3-e0a9f05fe156") : secret "monitoring-plugin-cert" not found Jul 06 19:26:25.671720 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.671626 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llznx" event={"ID":"85e87373-0d72-46c5-ae18-6c4af955b811","Type":"ContainerStarted","Data":"3b38dd8f14e05efff410f2503cf453cf811484973d23574e8d6ef390b58a68a6"} Jul 06 19:26:25.692201 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.692170 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:25.798088 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.798050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:25.894100 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:25.894062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:26.026383 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.026245 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:26:26.028941 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:26.028896 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf154b483_5d69_404a_ba2e_81e7eefa4385.slice/crio-7e7489c2017b5d0f27053bc12b575ad8a5fac164f67ab6ce51a5a0d147dfc6d0 WatchSource:0}: Error finding container 7e7489c2017b5d0f27053bc12b575ad8a5fac164f67ab6ce51a5a0d147dfc6d0: Status 404 returned error can't find the container with id 7e7489c2017b5d0f27053bc12b575ad8a5fac164f67ab6ce51a5a0d147dfc6d0 Jul 06 19:26:26.584804 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.584772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:26.587867 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.587838 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9e7d964-7d87-4b1f-83e3-e0a9f05fe156-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7vt7b\" (UID: \"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:26.624911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.624809 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:26.683550 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.683511 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"a9d934e35b01ecc1d5098df87f6d3a3babcb30ac3548003e626af0f6cccce5c8"} Jul 06 19:26:26.683741 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.683637 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"3ba7663921ba76536b29d6945750095c08ab2c0dab6c43043ec076b20639f7b2"} Jul 06 19:26:26.683741 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.683685 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"9c8d44ccbb00f6c5928ce06162b4395f92e83ceead44fe391bf6af3a1037114c"} Jul 06 19:26:26.683741 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.683725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"062eb333d8901de2a6f54a8a9dc553dd9fca4add56511d85e6c53277629829cd"} Jul 06 19:26:26.683907 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.683761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"db165fe3cb4d7279eae48dfd893975965614ed0e253fbced551a8d8d7504166d"} Jul 06 19:26:26.685830 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.685773 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llznx" event={"ID":"85e87373-0d72-46c5-ae18-6c4af955b811","Type":"ContainerStarted","Data":"0f1a0f7c07b029a509417da2d67d063969c85a094e1a7bca254b92fd3733bb7f"} Jul 06 19:26:26.685972 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.685893 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-llznx" Jul 06 19:26:26.690842 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.690809 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"6593b5bad4f76fd3ce20d0e14c07a61edfb0a7b10d992ab294d54e5be52bf7d2"} Jul 06 19:26:26.690975 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.690892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"e5347d026166884fe13bd5194d63cc48232b92c66fc38172cb18239f4438cac7"} Jul 06 19:26:26.690975 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.690945 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"ad9afacfde9140514008398b125a23bb89f0360dc2b4718edb7ceaa0fb26c169"} Jul 06 19:26:26.692700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.692672 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744bd578-d7pr7" event={"ID":"f154b483-5d69-404a-ba2e-81e7eefa4385","Type":"ContainerStarted","Data":"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203"} Jul 06 19:26:26.692808 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.692716 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744bd578-d7pr7" event={"ID":"f154b483-5d69-404a-ba2e-81e7eefa4385","Type":"ContainerStarted","Data":"7e7489c2017b5d0f27053bc12b575ad8a5fac164f67ab6ce51a5a0d147dfc6d0"} Jul 06 19:26:26.717527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.717474 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-llznx" podStartSLOduration=130.97394106 podStartE2EDuration="2m12.717453521s" totalImagesPullingTime="1.743512461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:26.714675108 +0000 UTC m=+168.220273591" watchObservedRunningTime="2026-07-06 19:26:26.717453521 +0000 UTC m=+168.223052005" Jul 06 19:26:26.743444 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.742639 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744bd578-d7pr7" podStartSLOduration=2.742616379 podStartE2EDuration="2.742616379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:26.740838414 +0000 UTC m=+168.246436917" watchObservedRunningTime="2026-07-06 19:26:26.742616379 +0000 UTC m=+168.248214863" Jul 06 19:26:26.889585 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.887875 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:26:26.894161 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.894139 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.896357 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.896328 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:26.898621 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.898584 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:26:26.899653 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.899586 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e66nq2vaiuiu8\"" Jul 06 19:26:26.900196 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900175 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:26:26.900344 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900212 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:26:26.900432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900370 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:26:26.900432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900386 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:26:26.900432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900409 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:26:26.900432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900419 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900766 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gv466\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900797 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900810 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900858 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.900975 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:26:26.901125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.901029 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:26:26.905512 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.905420 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:26:26.916866 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.916845 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:26:26.933012 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.932984 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:26:26.938848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.937170 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b"] Jul 06 19:26:26.939872 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.939850 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:26:26.988054 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988025 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988169 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988169 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988083 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988169 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988170 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988239 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988328 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988277 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988328 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988302 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988328 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988318 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988344 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988359 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988392 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988508 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988419 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9kd\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-kube-api-access-5l9kd\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988508 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988437 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988508 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988564 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988620 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:26.988743 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:26.988627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.089704 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.089704 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089677 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9kd\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-kube-api-access-5l9kd\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.089704 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089697 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089728 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089751 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089771 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089949 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.089994 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090103 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090066 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090214 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090369 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090392 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.090467 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090452 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.091122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.090994 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.091122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.091082 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.091718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.091334 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.092216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.092190 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.092885 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.092863 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.093045 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.093018 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.093115 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.093072 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.093356 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.093331 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.093708 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.093660 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.094691 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.094247 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.094800 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.094717 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.095130 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.095079 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.095884 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.096584 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.096760 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.096843 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.096876 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.097519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.097416 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.098068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.098045 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-config\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.107749 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.107727 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9kd\" (UniqueName: \"kubernetes.io/projected/dd4a7fe8-4ea1-471f-9218-507e7f9e58b9-kube-api-access-5l9kd\") pod \"prometheus-k8s-0\" (UID: \"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.225095 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.225055 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:27.373995 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.373971 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:26:27.375788 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:26:27.375767 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4a7fe8_4ea1_471f_9218_507e7f9e58b9.slice/crio-718d693b96161516c2310234cd0ac9a83ce1146261e030b14f66b0d97e5cfd38 WatchSource:0}: Error finding container 718d693b96161516c2310234cd0ac9a83ce1146261e030b14f66b0d97e5cfd38: Status 404 returned error can't find the container with id 718d693b96161516c2310234cd0ac9a83ce1146261e030b14f66b0d97e5cfd38 Jul 06 19:26:27.702677 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.702639 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"787c6e3a53927402e3153bd0ae408d94abcbcc86c37634d49411913202750dbd"} Jul 06 19:26:27.703122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.702715 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"3361380be60fb1dbf35a3f7862975bde71705f374f3281095a7b10f658490073"} Jul 06 19:26:27.703122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.702760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" event={"ID":"c381d033-1248-471e-b89b-edfd4a7c490b","Type":"ContainerStarted","Data":"dc51bff9b1b81999ebc26942778a132482c17738742aa63cce25058908547b58"} Jul 06 19:26:27.703122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.702868 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:27.704551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.704522 2555 generic.go:356] "Generic (PLEG): container finished" podID="dd4a7fe8-4ea1-471f-9218-507e7f9e58b9" containerID="5cfeb1e6a340e521375053dba889f8231d32b2c8e0b8b36401b49cf45f05c27a" exitCode=0 Jul 06 19:26:27.704733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.704625 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerDied","Data":"5cfeb1e6a340e521375053dba889f8231d32b2c8e0b8b36401b49cf45f05c27a"} Jul 06 19:26:27.704733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.704701 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"718d693b96161516c2310234cd0ac9a83ce1146261e030b14f66b0d97e5cfd38"} Jul 06 19:26:27.706009 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.705986 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" event={"ID":"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156","Type":"ContainerStarted","Data":"69c4cfe51d29fa465aeb5d33a3f95d8a0a51f0bb2ab24289332f812ad8806176"} Jul 06 19:26:27.709990 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.709967 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"026bde0f-1fa4-4925-9ac4-c1955a16c0c7","Type":"ContainerStarted","Data":"53e7c986b593a0bb189f471a26ebf396b443df4a0c11509b42315989d7ef4459"} Jul 06 19:26:27.729578 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.729520 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" podStartSLOduration=1.842028099 podStartE2EDuration="4.729501247s" totalImagesPullingTime="2.887473148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:27.728699614 +0000 UTC m=+169.234298115" watchObservedRunningTime="2026-07-06 19:26:27.729501247 +0000 UTC m=+169.235099730" Jul 06 19:26:27.825693 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:27.825633 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.643417579 podStartE2EDuration="6.825585722s" totalImagesPullingTime="4.160787967s" totalInitContainerRuntime="21.380176ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:27.823826325 +0000 UTC m=+169.329424808" watchObservedRunningTime="2026-07-06 19:26:27.825585722 +0000 UTC m=+169.331184205" Jul 06 19:26:28.500419 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.500377 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:28.570310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.570271 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:28.715478 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.715442 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" event={"ID":"f9e7d964-7d87-4b1f-83e3-e0a9f05fe156","Type":"ContainerStarted","Data":"32cbaaa11e88f85658ba3e373739b908161998cb97572e887f209764d01cd3c8"} Jul 06 19:26:28.716002 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.715536 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:28.720748 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.720726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" Jul 06 19:26:28.740530 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:28.740491 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7vt7b" podStartSLOduration=3.513755228 podStartE2EDuration="4.740478696s" totalImagesPullingTime="1.226723468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:28.738462089 +0000 UTC m=+170.244060572" watchObservedRunningTime="2026-07-06 19:26:28.740478696 +0000 UTC m=+170.246077178" Jul 06 19:26:30.405331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:30.405255 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.481044 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:30.481012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:31.734007 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.733970 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"c4e995b7d33434a59539be04d071b3c5a4d3aa290a99c85db52435e9849e30e6"} Jul 06 19:26:31.734491 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.734052 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"427ad22b9b763a9d49f108e5ea23cea839551dfe511468ca2d68186a8d93e4ed"} Jul 06 19:26:31.734491 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.734096 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"3c26e46d1aa6a5ff645e883840e4e3b7d9533c2d94710fe4f7f7feaadea7709e"} Jul 06 19:26:31.734491 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.734138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"dd5daae039a8cf772f8644564473987eedb8e3656803c6035a60b4c4bef8ad0c"} Jul 06 19:26:31.734491 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.734177 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"590eaa16e8951a75463125f3f60f093cc92e6be5bcb2632d7745c33fd442eed4"} Jul 06 19:26:31.734491 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.734225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd4a7fe8-4ea1-471f-9218-507e7f9e58b9","Type":"ContainerStarted","Data":"7ae25001b8f4d978395829bc9b82af90741eafe28f4344fac964acf6767ad87d"} Jul 06 19:26:31.821679 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:31.821585 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.476722137 podStartE2EDuration="5.821567349s" totalImagesPullingTime="3.320245544s" totalInitContainerRuntime="24.599668ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:31.820194498 +0000 UTC m=+173.325793015" watchObservedRunningTime="2026-07-06 19:26:31.821567349 +0000 UTC m=+173.327165832" Jul 06 19:26:32.225717 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:32.225685 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:26:32.969030 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:32.968998 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:26:33.105535 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:33.105496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.211212 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:33.211175 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.379139 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:33.379052 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.482887 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:33.482845 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.722557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:33.722534 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c7c865454-hxnnh" Jul 06 19:26:35.094730 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:35.094695 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:35.095085 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:35.094756 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:35.099475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:35.099454 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:35.756097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:35.756070 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:26:36.712503 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:36.712470 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-llznx" Jul 06 19:26:36.780980 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:36.780950 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:26:36.984966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:36.984876 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:37.085220 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:37.085188 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:37.150555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:37.150528 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:37.584324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:37.584288 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:37.687181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:37.687145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:45.397351 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:45.397309 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:45.463795 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:45.463761 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:45.925473 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:45.925442 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:46.723660 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:46.723624 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-577c68df98-9lq4r" podUID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" containerName="console" containerID="cri-o://5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6" gracePeriod=15 Jul 06 19:26:47.484317 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.484291 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:47.584189 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584155 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584431 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584198 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjpwk\" (UniqueName: \"kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584431 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584222 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584431 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584253 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584431 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584373 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584693 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584543 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert\") pod \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\" (UID: \"1cbd82ee-6d4f-46d9-9316-72be5cd05b73\") " Jul 06 19:26:47.584693 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584580 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca" (OuterVolumeSpecName: "service-ca") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:47.584792 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584748 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config" (OuterVolumeSpecName: "console-config") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:47.585049 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584921 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.585049 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584935 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:47.585049 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.584943 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.586636 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.586587 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:47.587113 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.587089 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:47.587113 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.587102 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk" (OuterVolumeSpecName: "kube-api-access-pjpwk") pod "1cbd82ee-6d4f-46d9-9316-72be5cd05b73" (UID: "1cbd82ee-6d4f-46d9-9316-72be5cd05b73"). InnerVolumeSpecName "kube-api-access-pjpwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:47.686415 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.686328 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-oauth-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.686415 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.686361 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjpwk\" (UniqueName: \"kubernetes.io/projected/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-kube-api-access-pjpwk\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.686415 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.686371 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-oauth-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.686415 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.686379 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cbd82ee-6d4f-46d9-9316-72be5cd05b73-console-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.801194 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.801163 2555 generic.go:356] "Generic (PLEG): container finished" podID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" containerID="5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6" exitCode=0 Jul 06 19:26:47.801630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.801203 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577c68df98-9lq4r" event={"ID":"1cbd82ee-6d4f-46d9-9316-72be5cd05b73","Type":"ContainerDied","Data":"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6"} Jul 06 19:26:47.801630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.801224 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577c68df98-9lq4r" Jul 06 19:26:47.801630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.801243 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577c68df98-9lq4r" event={"ID":"1cbd82ee-6d4f-46d9-9316-72be5cd05b73","Type":"ContainerDied","Data":"7ac87ce8d7ca7d2f1c7ee92fc90585ed8ed1ca3123517341f4c738d276eaff56"} Jul 06 19:26:47.801630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.801262 2555 scope.go:117] "RemoveContainer" containerID="5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6" Jul 06 19:26:47.816010 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.815992 2555 scope.go:117] "RemoveContainer" containerID="5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6" Jul 06 19:26:47.816285 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:26:47.816266 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6\": container with ID starting with 5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6 not found: ID does not exist" containerID="5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6" Jul 06 19:26:47.816341 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.816291 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6"} err="failed to get container status \"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6\": rpc error: code = NotFound desc = could not find container \"5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6\": container with ID starting with 5960c9533c8ea4d530076c2bc6c8efe28f03e52119a042418773cc26530d2fc6 not found: ID does not exist" Jul 06 19:26:47.830933 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.830905 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:47.845789 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:47.845762 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-577c68df98-9lq4r"] Jul 06 19:26:48.967935 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:48.967901 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" path="/var/lib/kubelet/pods/1cbd82ee-6d4f-46d9-9316-72be5cd05b73/volumes" Jul 06 19:26:52.288024 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:52.287926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:52.824313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:52.824278 2555 generic.go:356] "Generic (PLEG): container finished" podID="c197b891-612c-4ba1-9cad-abe34cad683b" containerID="a4914dfed9bb2f809d9471c32a3c2b7ef5f49f889f9976ac84e1c77cad9c349a" exitCode=0 Jul 06 19:26:52.824490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:52.824357 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" event={"ID":"c197b891-612c-4ba1-9cad-abe34cad683b","Type":"ContainerDied","Data":"a4914dfed9bb2f809d9471c32a3c2b7ef5f49f889f9976ac84e1c77cad9c349a"} Jul 06 19:26:52.824729 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:52.824714 2555 scope.go:117] "RemoveContainer" containerID="a4914dfed9bb2f809d9471c32a3c2b7ef5f49f889f9976ac84e1c77cad9c349a" Jul 06 19:26:53.045964 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:53.045926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:53.124321 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:53.124227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:53.830117 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:26:53.830080 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-d865r" event={"ID":"c197b891-612c-4ba1-9cad-abe34cad683b","Type":"ContainerStarted","Data":"dbca78d653f3f74986cae38d6c5dc019994dd71e308e25ec12132121eae98431"} Jul 06 19:27:00.404531 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:00.404496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:00.474447 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:00.474385 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:01.808694 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:01.808658 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-798dc8955f-m59c2" podUID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" containerName="console" containerID="cri-o://d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99" gracePeriod=15 Jul 06 19:27:02.061740 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.061683 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:27:02.115660 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115632 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.115843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115669 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.115843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115702 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.115843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115731 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c4f8\" (UniqueName: \"kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.115843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115746 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.115843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115782 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.116109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.115851 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca\") pod \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\" (UID: \"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96\") " Jul 06 19:27:02.116109 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.116050 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config" (OuterVolumeSpecName: "console-config") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:02.116208 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.116130 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:02.116361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.116331 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:02.116446 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.116392 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:02.118191 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.118171 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8" (OuterVolumeSpecName: "kube-api-access-8c4f8") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "kube-api-access-8c4f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:02.118286 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.118186 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:27:02.118328 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.118286 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" (UID: "7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:27:02.216494 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216466 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-trusted-ca-bundle\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216494 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216492 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216494 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216501 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216715 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216510 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-oauth-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216715 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216518 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216715 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216526 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c4f8\" (UniqueName: \"kubernetes.io/projected/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-kube-api-access-8c4f8\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.216715 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.216534 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96-console-oauth-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:27:02.866606 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.866563 2555 generic.go:356] "Generic (PLEG): container finished" podID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" containerID="d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99" exitCode=0 Jul 06 19:27:02.866998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.866622 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798dc8955f-m59c2" Jul 06 19:27:02.866998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.866647 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798dc8955f-m59c2" event={"ID":"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96","Type":"ContainerDied","Data":"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99"} Jul 06 19:27:02.866998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.866697 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798dc8955f-m59c2" event={"ID":"7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96","Type":"ContainerDied","Data":"9312c664ca7f7105465e7bbbc3191ebe527513248b457e957832b7dfed9c34f5"} Jul 06 19:27:02.866998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.866717 2555 scope.go:117] "RemoveContainer" containerID="d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99" Jul 06 19:27:02.878341 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.878320 2555 scope.go:117] "RemoveContainer" containerID="d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99" Jul 06 19:27:02.878913 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:27:02.878892 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99\": container with ID starting with d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99 not found: ID does not exist" containerID="d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99" Jul 06 19:27:02.878989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.878926 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99"} err="failed to get container status \"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99\": rpc error: code = NotFound desc = could not find container \"d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99\": container with ID starting with d297f6f91c589fe8de78db39211843f38223529e6cd4963f38ea2c8e3d89bf99 not found: ID does not exist" Jul 06 19:27:02.895247 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.895224 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:27:02.898174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.898150 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-798dc8955f-m59c2"] Jul 06 19:27:02.966572 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:02.966538 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" path="/var/lib/kubelet/pods/7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96/volumes" Jul 06 19:27:25.064749 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:25.064720 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:25.132499 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:25.132466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:27.225369 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:27.225342 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:27:27.241557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:27.241531 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:27:27.988843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:27.988815 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:27:30.282923 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:30.282890 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:30.367982 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:30.367954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:39.940331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:39.940290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:40.398111 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:40.398079 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:42.533709 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:42.533678 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:49.755437 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:49.755397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:27:49.757753 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:49.757734 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22353e76-49ad-4dda-be56-955c247e7cae-metrics-certs\") pod \"network-metrics-daemon-vphct\" (UID: \"22353e76-49ad-4dda-be56-955c247e7cae\") " pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:27:49.772756 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:49.772735 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mm8d\"" Jul 06 19:27:49.780330 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:49.780317 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vphct" Jul 06 19:27:49.911383 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:49.911360 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vphct"] Jul 06 19:27:49.914096 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:27:49.914065 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22353e76_49ad_4dda_be56_955c247e7cae.slice/crio-b4844dcc6c8737fa5b77e806dac991d8074b9e9561b64a33d6f2e604427546e4 WatchSource:0}: Error finding container b4844dcc6c8737fa5b77e806dac991d8074b9e9561b64a33d6f2e604427546e4: Status 404 returned error can't find the container with id b4844dcc6c8737fa5b77e806dac991d8074b9e9561b64a33d6f2e604427546e4 Jul 06 19:27:50.060087 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.060011 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vphct" event={"ID":"22353e76-49ad-4dda-be56-955c247e7cae","Type":"ContainerStarted","Data":"b4844dcc6c8737fa5b77e806dac991d8074b9e9561b64a33d6f2e604427546e4"} Jul 06 19:27:50.998630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.998544 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:27:50.999639 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999562 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" containerName="console" Jul 06 19:27:50.999639 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999621 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" containerName="console" Jul 06 19:27:50.999874 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999666 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" containerName="console" Jul 06 19:27:50.999874 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999675 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" containerName="console" Jul 06 19:27:50.999874 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999753 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2be36e-6dfa-4ac3-9b9d-bf067c91bf96" containerName="console" Jul 06 19:27:50.999874 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:50.999767 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbd82ee-6d4f-46d9-9316-72be5cd05b73" containerName="console" Jul 06 19:27:51.003018 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.003000 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.020747 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.020721 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:27:51.057539 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.057495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.067499 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.067469 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vphct" event={"ID":"22353e76-49ad-4dda-be56-955c247e7cae","Type":"ContainerStarted","Data":"0e8b3b4ad77e24c2515ef6d475e8069e22de078275aab670739080adec97c448"} Jul 06 19:27:51.067640 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.067530 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vphct" event={"ID":"22353e76-49ad-4dda-be56-955c247e7cae","Type":"ContainerStarted","Data":"a2daa630a8a2f986dd0fd7f2de848e25cf8e84ebad7f200e07f60240f8ad57bf"} Jul 06 19:27:51.084782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.084718 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vphct" podStartSLOduration=251.15456566 podStartE2EDuration="4m12.084697645s" totalImagesPullingTime="930.131985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:51.083431964 +0000 UTC m=+252.589030447" watchObservedRunningTime="2026-07-06 19:27:51.084697645 +0000 UTC m=+252.590296128" Jul 06 19:27:51.138116 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.138014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.165737 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165703 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52ddm\" (UniqueName: \"kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165737 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165742 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165763 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165827 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165891 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.165949 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.165935 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.239871 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.239837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.266806 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.266977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.266977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266838 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.266977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266874 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.266977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.266977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.266929 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.267271 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.267019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52ddm\" (UniqueName: \"kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.267851 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.267800 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.267851 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.267833 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.267991 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.267810 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.267991 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.267800 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.269457 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.269438 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.269541 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.269457 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.277130 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.277101 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52ddm\" (UniqueName: \"kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm\") pod \"console-795f6cb7b5-llsrj\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.330976 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.330943 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:27:51.341180 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.341150 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.481972 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:51.481695 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:27:51.485347 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:27:51.485307 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea04865_616c_4054_bf68_468a7a42e83f.slice/crio-03379f802b417efdf922f23ef8c1e27bbae0b3fb8347ac2ff4841161aa326d40 WatchSource:0}: Error finding container 03379f802b417efdf922f23ef8c1e27bbae0b3fb8347ac2ff4841161aa326d40: Status 404 returned error can't find the container with id 03379f802b417efdf922f23ef8c1e27bbae0b3fb8347ac2ff4841161aa326d40 Jul 06 19:27:52.075308 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:52.075272 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cb7b5-llsrj" event={"ID":"dea04865-616c-4054-bf68-468a7a42e83f","Type":"ContainerStarted","Data":"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759"} Jul 06 19:27:52.075778 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:52.075322 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cb7b5-llsrj" event={"ID":"dea04865-616c-4054-bf68-468a7a42e83f","Type":"ContainerStarted","Data":"03379f802b417efdf922f23ef8c1e27bbae0b3fb8347ac2ff4841161aa326d40"} Jul 06 19:27:52.096316 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:52.096265 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-795f6cb7b5-llsrj" podStartSLOduration=2.096250869 podStartE2EDuration="2.096250869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:52.094912093 +0000 UTC m=+253.600510574" watchObservedRunningTime="2026-07-06 19:27:52.096250869 +0000 UTC m=+253.601849359" Jul 06 19:27:55.320284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:55.320256 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:55.410038 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:55.410004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:56.815983 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:56.815945 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:27:56.857505 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:56.857471 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:27:56.861252 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:56.861233 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:56.876702 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:56.876682 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:27:57.020583 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020583 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020605 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020633 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020669 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2wws\" (UniqueName: \"kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.020809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.020781 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.121967 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122004 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2wws\" (UniqueName: \"kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122101 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122284 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.122977 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122939 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.123083 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.122958 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.123165 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.123144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.123223 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.123206 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.124680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.124660 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.124775 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.124676 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.131167 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.131139 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2wws\" (UniqueName: \"kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws\") pod \"console-6d755c5c45-54vxb\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.163725 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.163696 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:57.170734 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.170710 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:27:57.265386 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.265335 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:57.301680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:57.301658 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:27:57.304394 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:27:57.304372 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d778533_a4db_4747_b796_a2ae29e5694f.slice/crio-9b4238527a6b1bc6e19e34225708a905c8c52d883f19cdf2b4562b5757035762 WatchSource:0}: Error finding container 9b4238527a6b1bc6e19e34225708a905c8c52d883f19cdf2b4562b5757035762: Status 404 returned error can't find the container with id 9b4238527a6b1bc6e19e34225708a905c8c52d883f19cdf2b4562b5757035762 Jul 06 19:27:58.101051 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:58.101019 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d755c5c45-54vxb" event={"ID":"6d778533-a4db-4747-b796-a2ae29e5694f","Type":"ContainerStarted","Data":"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70"} Jul 06 19:27:58.101416 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:27:58.101074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d755c5c45-54vxb" event={"ID":"6d778533-a4db-4747-b796-a2ae29e5694f","Type":"ContainerStarted","Data":"9b4238527a6b1bc6e19e34225708a905c8c52d883f19cdf2b4562b5757035762"} Jul 06 19:28:01.332015 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:01.331938 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:28:07.171742 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.171713 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:28:07.172276 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.171792 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:28:07.176294 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.176272 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:28:07.198227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.198175 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d755c5c45-54vxb" podStartSLOduration=11.19816048 podStartE2EDuration="11.19816048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:58.136482349 +0000 UTC m=+259.642080832" watchObservedRunningTime="2026-07-06 19:28:07.19816048 +0000 UTC m=+268.703758961" Jul 06 19:28:07.301262 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.301232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:07.367979 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:07.367950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:08.144356 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:08.144331 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:28:08.216074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:08.216037 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:28:08.394563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:08.394477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:08.496041 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:08.496001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:16.417887 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:16.417846 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:16.521884 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:16.521850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:16.617071 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:16.617042 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:21.836445 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:21.836383 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-795f6cb7b5-llsrj" podUID="dea04865-616c-4054-bf68-468a7a42e83f" containerName="console" containerID="cri-o://22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759" gracePeriod=15 Jul 06 19:28:22.100951 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.100892 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:28:22.137303 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137273 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137335 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137365 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137395 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137422 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52ddm\" (UniqueName: \"kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137490 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137456 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137498 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config\") pod \"dea04865-616c-4054-bf68-468a7a42e83f\" (UID: \"dea04865-616c-4054-bf68-468a7a42e83f\") " Jul 06 19:28:22.137833 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137766 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca" (OuterVolumeSpecName: "service-ca") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:22.137833 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137789 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:22.137994 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.137952 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:22.138120 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.138005 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config" (OuterVolumeSpecName: "console-config") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:22.138179 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.138166 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-oauth-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.138225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.138183 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.138225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.138196 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-trusted-ca-bundle\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.138225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.138209 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dea04865-616c-4054-bf68-468a7a42e83f-console-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.140299 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.140273 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:28:22.140299 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.140288 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm" (OuterVolumeSpecName: "kube-api-access-52ddm") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "kube-api-access-52ddm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:28:22.140448 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.140290 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dea04865-616c-4054-bf68-468a7a42e83f" (UID: "dea04865-616c-4054-bf68-468a7a42e83f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:28:22.199774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.199742 2555 generic.go:356] "Generic (PLEG): container finished" podID="dea04865-616c-4054-bf68-468a7a42e83f" containerID="22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759" exitCode=0 Jul 06 19:28:22.199918 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.199802 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cb7b5-llsrj" Jul 06 19:28:22.199918 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.199800 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cb7b5-llsrj" event={"ID":"dea04865-616c-4054-bf68-468a7a42e83f","Type":"ContainerDied","Data":"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759"} Jul 06 19:28:22.200099 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.199934 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cb7b5-llsrj" event={"ID":"dea04865-616c-4054-bf68-468a7a42e83f","Type":"ContainerDied","Data":"03379f802b417efdf922f23ef8c1e27bbae0b3fb8347ac2ff4841161aa326d40"} Jul 06 19:28:22.200099 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.199965 2555 scope.go:117] "RemoveContainer" containerID="22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759" Jul 06 19:28:22.208924 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.208910 2555 scope.go:117] "RemoveContainer" containerID="22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759" Jul 06 19:28:22.209153 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:28:22.209136 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759\": container with ID starting with 22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759 not found: ID does not exist" containerID="22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759" Jul 06 19:28:22.209195 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.209161 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759"} err="failed to get container status \"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759\": rpc error: code = NotFound desc = could not find container \"22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759\": container with ID starting with 22e5dc50744c352fdc43b676d9dcc377a754261459c9387ad3c9c5de9cc6c759 not found: ID does not exist" Jul 06 19:28:22.223737 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.223714 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:28:22.226278 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.226259 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-795f6cb7b5-llsrj"] Jul 06 19:28:22.239351 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.239331 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.239351 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.239351 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dea04865-616c-4054-bf68-468a7a42e83f-console-oauth-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.239473 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.239361 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52ddm\" (UniqueName: \"kubernetes.io/projected/dea04865-616c-4054-bf68-468a7a42e83f-kube-api-access-52ddm\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:22.966772 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:22.966730 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dea04865-616c-4054-bf68-468a7a42e83f" path="/var/lib/kubelet/pods/dea04865-616c-4054-bf68-468a7a42e83f/volumes" Jul 06 19:28:33.236274 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.236216 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-744bd578-d7pr7" podUID="f154b483-5d69-404a-ba2e-81e7eefa4385" containerName="console" containerID="cri-o://90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203" gracePeriod=15 Jul 06 19:28:33.483956 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.483933 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:28:33.536894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.536813 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.536894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.536869 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537096 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.536900 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537096 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.536928 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537096 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.536950 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537096 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537004 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gbpj\" (UniqueName: \"kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537301 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537182 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config\") pod \"f154b483-5d69-404a-ba2e-81e7eefa4385\" (UID: \"f154b483-5d69-404a-ba2e-81e7eefa4385\") " Jul 06 19:28:33.537476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537421 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:33.537476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537441 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca" (OuterVolumeSpecName: "service-ca") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:33.537476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537462 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:33.537768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537496 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config" (OuterVolumeSpecName: "console-config") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:28:33.537768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537535 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.537768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.537555 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-oauth-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.539709 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.539682 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:28:33.539795 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.539756 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:28:33.539795 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.539768 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj" (OuterVolumeSpecName: "kube-api-access-8gbpj") pod "f154b483-5d69-404a-ba2e-81e7eefa4385" (UID: "f154b483-5d69-404a-ba2e-81e7eefa4385"). InnerVolumeSpecName "kube-api-access-8gbpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:28:33.638142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.638091 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-trusted-ca-bundle\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.638142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.638139 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f154b483-5d69-404a-ba2e-81e7eefa4385-console-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.638142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.638150 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gbpj\" (UniqueName: \"kubernetes.io/projected/f154b483-5d69-404a-ba2e-81e7eefa4385-kube-api-access-8gbpj\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.638142 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.638158 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-oauth-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:33.638400 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:33.638168 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f154b483-5d69-404a-ba2e-81e7eefa4385-console-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:28:34.250871 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.250836 2555 generic.go:356] "Generic (PLEG): container finished" podID="f154b483-5d69-404a-ba2e-81e7eefa4385" containerID="90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203" exitCode=0 Jul 06 19:28:34.251256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.250884 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744bd578-d7pr7" Jul 06 19:28:34.251256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.250929 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744bd578-d7pr7" event={"ID":"f154b483-5d69-404a-ba2e-81e7eefa4385","Type":"ContainerDied","Data":"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203"} Jul 06 19:28:34.251256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.250985 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744bd578-d7pr7" event={"ID":"f154b483-5d69-404a-ba2e-81e7eefa4385","Type":"ContainerDied","Data":"7e7489c2017b5d0f27053bc12b575ad8a5fac164f67ab6ce51a5a0d147dfc6d0"} Jul 06 19:28:34.251256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.251010 2555 scope.go:117] "RemoveContainer" containerID="90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203" Jul 06 19:28:34.259549 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.259533 2555 scope.go:117] "RemoveContainer" containerID="90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203" Jul 06 19:28:34.259812 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:28:34.259794 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203\": container with ID starting with 90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203 not found: ID does not exist" containerID="90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203" Jul 06 19:28:34.259851 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.259822 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203"} err="failed to get container status \"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203\": rpc error: code = NotFound desc = could not find container \"90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203\": container with ID starting with 90e723df1ff38c057d840859a0e57cf874dafdcee4d81570ca9c45e2dbfe4203 not found: ID does not exist" Jul 06 19:28:34.275083 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.275060 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:28:34.284679 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.284656 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744bd578-d7pr7"] Jul 06 19:28:34.967301 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:34.967259 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f154b483-5d69-404a-ba2e-81e7eefa4385" path="/var/lib/kubelet/pods/f154b483-5d69-404a-ba2e-81e7eefa4385/volumes" Jul 06 19:28:37.637152 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:37.637092 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:37.738947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:37.738903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:38.816268 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:38.816242 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:28:58.952983 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:28:58.952950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:04.314323 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:04.314278 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:05.115175 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:05.115132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:07.945846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:07.945809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:08.010727 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:08.010699 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:08.920817 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:08.920781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:09.020897 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:09.020850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:09.430771 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:09.430731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:09.530928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:09.530895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:11.891013 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:11.890962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:12.048618 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:12.048557 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:14.728370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:14.728316 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:14.787947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:14.787917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:18.151356 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:18.151318 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:18.253821 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:18.253764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:19.062531 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:19.062501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:23.379991 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:23.379955 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:23.458828 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:23.458785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:25.335675 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:25.335633 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:32.122564 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:32.122478 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:32.185170 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:32.185138 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:41.599919 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:41.599887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:41.694758 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:41.694723 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:42.514897 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:42.514862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:42.617024 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:42.616986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.330738 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.330694 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.518635 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.518603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.619360 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.619280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.670263 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd"] Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.670996 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dea04865-616c-4054-bf68-468a7a42e83f" containerName="console" Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.671016 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dea04865-616c-4054-bf68-468a7a42e83f" containerName="console" Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.671050 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f154b483-5d69-404a-ba2e-81e7eefa4385" containerName="console" Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.671059 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f154b483-5d69-404a-ba2e-81e7eefa4385" containerName="console" Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.671203 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f154b483-5d69-404a-ba2e-81e7eefa4385" containerName="console" Jul 06 19:29:43.673331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.671215 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dea04865-616c-4054-bf68-468a7a42e83f" containerName="console" Jul 06 19:29:43.675290 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.675266 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.678019 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.677995 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:29:43.678019 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.678006 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:29:43.678427 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.678410 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:29:43.678611 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.678569 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bkc2r\"" Jul 06 19:29:43.678710 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.678439 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:29:43.693258 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.693236 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd"] Jul 06 19:29:43.780975 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.780927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.816230 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.816196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.816408 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.816249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.816480 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.816375 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scjhr\" (UniqueName: \"kubernetes.io/projected/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-kube-api-access-scjhr\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.816480 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.816469 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-manifests\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.816584 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.816503 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-tmp\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.882856 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.882762 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.917972 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.917938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.918090 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.917986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.918090 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.918048 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scjhr\" (UniqueName: \"kubernetes.io/projected/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-kube-api-access-scjhr\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.918090 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.918079 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-manifests\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.918251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.918105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-tmp\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.918489 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.918468 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-manifests\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.920477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.920457 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-tmp\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.920556 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.920544 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.920687 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.920672 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.928265 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.928246 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scjhr\" (UniqueName: \"kubernetes.io/projected/83bcb5f0-4604-4dcb-af4c-ef9b9f219685-kube-api-access-scjhr\") pod \"opendatahub-operator-controller-manager-7fcb8c7879-ztzgd\" (UID: \"83bcb5f0-4604-4dcb-af4c-ef9b9f219685\") " pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:43.981630 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.981558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:43.986548 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:43.986512 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:44.088462 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:44.088434 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:44.141037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:44.141013 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd"] Jul 06 19:29:44.143297 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:29:44.143272 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bcb5f0_4604_4dcb_af4c_ef9b9f219685.slice/crio-92a15f640f2c19f5be2b0f0fb6b5c2362370dde0c46610944432091a9aa5925f WatchSource:0}: Error finding container 92a15f640f2c19f5be2b0f0fb6b5c2362370dde0c46610944432091a9aa5925f: Status 404 returned error can't find the container with id 92a15f640f2c19f5be2b0f0fb6b5c2362370dde0c46610944432091a9aa5925f Jul 06 19:29:44.145105 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:44.145087 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:29:44.154507 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:44.154488 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:44.538580 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:44.538544 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerStarted","Data":"92a15f640f2c19f5be2b0f0fb6b5c2362370dde0c46610944432091a9aa5925f"} Jul 06 19:29:47.555740 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:47.555708 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerStarted","Data":"4020a2b1fd647250fb7c40a3c7754bdf6187e994fa81b99e5a56080667008d8c"} Jul 06 19:29:47.681980 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:47.681930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:47.777485 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:47.777460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:48.562244 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:48.562155 2555 generic.go:356] "Generic (PLEG): container finished" podID="83bcb5f0-4604-4dcb-af4c-ef9b9f219685" containerID="4020a2b1fd647250fb7c40a3c7754bdf6187e994fa81b99e5a56080667008d8c" exitCode=0 Jul 06 19:29:48.562673 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:48.562245 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerDied","Data":"4020a2b1fd647250fb7c40a3c7754bdf6187e994fa81b99e5a56080667008d8c"} Jul 06 19:29:49.568986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:49.568945 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerStarted","Data":"dea4d65dd3058415ee83942af5cc65dd3204651b0aa95e9f64cefdad27051619"} Jul 06 19:29:49.569466 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:49.569031 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:29:49.594076 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:49.594022 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" podStartSLOduration=2.243331999 podStartE2EDuration="6.594005507s" totalImagesPullingTime="3.336853878s" totalInitContainerRuntime="1.01381963s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:49.592183609 +0000 UTC m=+371.097782091" watchObservedRunningTime="2026-07-06 19:29:49.594005507 +0000 UTC m=+371.099603988" Jul 06 19:29:55.756457 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:55.756420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:55.858131 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:29:55.858103 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:00.575898 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:00.575871 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:30:03.765104 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:03.765068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:05.536709 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:05.536675 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:12.876082 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:12.876045 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:14.381967 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:14.381904 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:14.735194 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:14.735156 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:14.881524 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:14.881491 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:14.972896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:14.972867 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:16.236312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:16.236276 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:19.523833 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:19.523785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:19.624471 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:19.624386 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:20.425987 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:20.425954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:20.692706 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:20.692581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:20.795060 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:20.794999 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:22.387290 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.387228 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg"] Jul 06 19:30:22.395810 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.395785 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.398948 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.398928 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:30:22.400361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.400343 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:30:22.400637 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.400610 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:30:22.400637 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.400631 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:30:22.400787 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.400647 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-s6bll\"" Jul 06 19:30:22.400787 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.400687 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:30:22.406721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.406700 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg"] Jul 06 19:30:22.451197 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.451156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-metrics-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.451371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.451238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a29180a4-64ad-4de7-8080-8b974bed1a5f-manager-config\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.451371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.451267 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpm2g\" (UniqueName: \"kubernetes.io/projected/a29180a4-64ad-4de7-8080-8b974bed1a5f-kube-api-access-bpm2g\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.451371 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.451298 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.551959 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.551925 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-metrics-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.552147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.551976 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a29180a4-64ad-4de7-8080-8b974bed1a5f-manager-config\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.552147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.551999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpm2g\" (UniqueName: \"kubernetes.io/projected/a29180a4-64ad-4de7-8080-8b974bed1a5f-kube-api-access-bpm2g\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.552147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.552019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.552703 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.552670 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a29180a4-64ad-4de7-8080-8b974bed1a5f-manager-config\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.554864 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.554842 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-metrics-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.554925 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.554842 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a29180a4-64ad-4de7-8080-8b974bed1a5f-cert\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.571501 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.571479 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpm2g\" (UniqueName: \"kubernetes.io/projected/a29180a4-64ad-4de7-8080-8b974bed1a5f-kube-api-access-bpm2g\") pod \"lws-controller-manager-cf88bd784-wb2lg\" (UID: \"a29180a4-64ad-4de7-8080-8b974bed1a5f\") " pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.591162 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.591142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:22.678110 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.678008 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:22.704994 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.704956 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:22.866409 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:22.866368 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg"] Jul 06 19:30:22.870234 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:30:22.870212 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29180a4_64ad_4de7_8080_8b974bed1a5f.slice/crio-c945f2f51dfeef7fb5cb877199cc5afa5ecb7e2fb522a5acd8a03088e12d7950 WatchSource:0}: Error finding container c945f2f51dfeef7fb5cb877199cc5afa5ecb7e2fb522a5acd8a03088e12d7950: Status 404 returned error can't find the container with id c945f2f51dfeef7fb5cb877199cc5afa5ecb7e2fb522a5acd8a03088e12d7950 Jul 06 19:30:23.049760 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.049727 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r"] Jul 06 19:30:23.054361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.054340 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.058246 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.058226 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:30:23.058478 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.058464 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:30:23.058666 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.058654 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2qk76\"" Jul 06 19:30:23.059242 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.059221 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:30:23.059354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.059228 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:30:23.083707 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.083679 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r"] Jul 06 19:30:23.158069 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158028 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvcb\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-kube-api-access-wpvcb\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158239 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158087 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158239 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158239 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158374 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158281 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158374 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158328 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d490081a-11d5-45c5-a3b3-539a58963421-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.158477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.158427 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259099 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259060 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259287 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259108 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259287 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259140 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d490081a-11d5-45c5-a3b3-539a58963421-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259287 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259191 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259287 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvcb\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-kube-api-access-wpvcb\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259498 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259288 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.259498 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.259320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.260079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.260052 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.261631 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.261584 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d490081a-11d5-45c5-a3b3-539a58963421-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.261736 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.261655 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.262031 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.261996 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.262069 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.262036 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d490081a-11d5-45c5-a3b3-539a58963421-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.273702 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.273675 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvcb\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-kube-api-access-wpvcb\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.273702 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.273697 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d490081a-11d5-45c5-a3b3-539a58963421-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9kk6r\" (UID: \"d490081a-11d5-45c5-a3b3-539a58963421\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.365063 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.364984 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:23.504496 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.504465 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r"] Jul 06 19:30:23.507401 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:30:23.507357 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd490081a_11d5_45c5_a3b3_539a58963421.slice/crio-fb851a7eb193b837aa752239ed004bca12b76ef823fd2373e38e87d4735f2ae3 WatchSource:0}: Error finding container fb851a7eb193b837aa752239ed004bca12b76ef823fd2373e38e87d4735f2ae3: Status 404 returned error can't find the container with id fb851a7eb193b837aa752239ed004bca12b76ef823fd2373e38e87d4735f2ae3 Jul 06 19:30:23.733322 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.733287 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" event={"ID":"a29180a4-64ad-4de7-8080-8b974bed1a5f","Type":"ContainerStarted","Data":"c945f2f51dfeef7fb5cb877199cc5afa5ecb7e2fb522a5acd8a03088e12d7950"} Jul 06 19:30:23.735042 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.735008 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" event={"ID":"d490081a-11d5-45c5-a3b3-539a58963421","Type":"ContainerStarted","Data":"fb851a7eb193b837aa752239ed004bca12b76ef823fd2373e38e87d4735f2ae3"} Jul 06 19:30:23.840136 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.840099 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:23.946004 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:23.945916 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.059387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:24.059247 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.157618 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:24.157552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.228224 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:24.228183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:25.235181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:25.235126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:25.332276 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:25.332232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:25.393907 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:25.393879 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:26.758030 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:26.757984 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" event={"ID":"a29180a4-64ad-4de7-8080-8b974bed1a5f","Type":"ContainerStarted","Data":"d09b0e53d60c77958d34377dcc12678936c677b16b52ddccaf08cef566821fbf"} Jul 06 19:30:26.758495 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:26.758303 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:26.791911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:26.791863 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" podStartSLOduration=1.985008313 podStartE2EDuration="4.791843665s" totalImagesPullingTime="2.806835352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:26.785534262 +0000 UTC m=+408.291132766" watchObservedRunningTime="2026-07-06 19:30:26.791843665 +0000 UTC m=+408.297442148" Jul 06 19:30:26.869621 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:26.869553 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:30:26.869740 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:26.869645 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:30:27.766957 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:27.766911 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" event={"ID":"d490081a-11d5-45c5-a3b3-539a58963421","Type":"ContainerStarted","Data":"7a74208d7653119fa6350e0ec57b98abed55532c511c5a2903d0f0b9b93d302e"} Jul 06 19:30:27.767493 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:27.767479 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:27.768912 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:27.768878 2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-9kk6r container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:30:27.769048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:27.768941 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" podUID="d490081a-11d5-45c5-a3b3-539a58963421" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:30:27.792929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:27.792879 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" podStartSLOduration=1.433750479 podStartE2EDuration="4.792862607s" totalImagesPullingTime="3.359112128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:27.791839449 +0000 UTC m=+409.297437933" watchObservedRunningTime="2026-07-06 19:30:27.792862607 +0000 UTC m=+409.298461089" Jul 06 19:30:28.771993 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:28.771962 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9kk6r" Jul 06 19:30:37.769333 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:37.769302 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:30:40.920501 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:30:40.920460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:14.293547 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:14.293456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:14.392528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:14.392496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:14.496148 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:14.496114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:15.209328 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:15.209300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:15.414391 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:15.414354 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:41.140098 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:41.140064 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:42.139740 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:42.139704 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:47.155829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:47.155787 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:49.401721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:49.401692 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:49.495954 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:49.495920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:52.479701 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:52.479665 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:53.161732 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:53.161695 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:55.574286 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:55.574250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:55.803140 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:55.803066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:58.178392 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:58.178327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:59.437793 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:59.437765 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:59.536253 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:59.536215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:59.746484 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:59.746455 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:59.822189 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:31:59.822166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:02.624011 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:02.623983 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:02.925335 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:02.925260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:03.377472 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:03.377439 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:03.460526 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:03.460490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:10.429990 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:10.429952 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:10.521621 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:10.521580 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:12.669714 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.669678 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259"] Jul 06 19:32:12.677038 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.677010 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.680210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.680181 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:32:12.680430 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.680414 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:32:12.680515 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.680458 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ft7g9\"" Jul 06 19:32:12.685487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.685463 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259"] Jul 06 19:32:12.817498 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.817460 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.817669 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.817515 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shr4t\" (UniqueName: \"kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.918310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.918273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.918503 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.918331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shr4t\" (UniqueName: \"kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.918652 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.918632 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.928781 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.928727 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shr4t\" (UniqueName: \"kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t\") pod \"kuadrant-operator-controller-manager-55467d9f7-6s259\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:12.988334 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:12.988310 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:13.126116 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.126094 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259"] Jul 06 19:32:13.128696 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:32:13.128667 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d06ce7_6ac2_4a0c_8d72_b238fa60ab7f.slice/crio-104a0e3b5d2cba0005aaeffb5cd9d803a8d8dc238f2bf669cbf57bdbdffc76ed WatchSource:0}: Error finding container 104a0e3b5d2cba0005aaeffb5cd9d803a8d8dc238f2bf669cbf57bdbdffc76ed: Status 404 returned error can't find the container with id 104a0e3b5d2cba0005aaeffb5cd9d803a8d8dc238f2bf669cbf57bdbdffc76ed Jul 06 19:32:13.188436 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.188374 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259"] Jul 06 19:32:13.193547 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.193524 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259"] Jul 06 19:32:13.211338 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.211317 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:13.215782 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.215767 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.231778 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.231756 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:13.322022 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.321985 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kzcl\" (UniqueName: \"kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.322164 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.322036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.422893 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.422862 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kzcl\" (UniqueName: \"kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.423026 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.422909 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.423352 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.423326 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.434686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.434666 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kzcl\" (UniqueName: \"kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl\") pod \"kuadrant-operator-controller-manager-55467d9f7-fm82f\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.529910 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.529884 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:13.676813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:13.676787 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:13.707849 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:32:13.707814 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75059e8_7c9b_41a4_908f_326fd3b292c6.slice/crio-32ab8d137ddeb68015d61fbc5e4ac61eab718264150aac93728a2055ce725707 WatchSource:0}: Error finding container 32ab8d137ddeb68015d61fbc5e4ac61eab718264150aac93728a2055ce725707: Status 404 returned error can't find the container with id 32ab8d137ddeb68015d61fbc5e4ac61eab718264150aac93728a2055ce725707 Jul 06 19:32:14.244187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:14.244156 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" event={"ID":"f75059e8-7c9b-41a4-908f-326fd3b292c6","Type":"ContainerStarted","Data":"32ab8d137ddeb68015d61fbc5e4ac61eab718264150aac93728a2055ce725707"} Jul 06 19:32:15.216567 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:15.216534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:15.298738 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:15.298706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:17.480685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:17.480642 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:17.579928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:17.579895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:18.961475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:18.961431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:19.062052 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.062012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:19.165075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.165055 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:19.266894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.266810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:19.272575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.272549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" event={"ID":"f75059e8-7c9b-41a4-908f-326fd3b292c6","Type":"ContainerStarted","Data":"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372"} Jul 06 19:32:19.272754 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.272731 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:19.274468 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.274439 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" containerName="manager" containerID="cri-o://6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e" gracePeriod=2 Jul 06 19:32:19.298061 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:19.298023 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6s259\" is forbidden: User \"system:node:ip-10-0-133-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-207.ec2.internal' and this object" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:19.298354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.298323 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" podStartSLOduration=0.867264672 podStartE2EDuration="6.298309602s" totalImagesPullingTime="5.43104493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:19.295586709 +0000 UTC m=+520.801185192" watchObservedRunningTime="2026-07-06 19:32:19.298309602 +0000 UTC m=+520.803908084" Jul 06 19:32:19.516777 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.516750 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:19.519343 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:19.519299 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6s259\" is forbidden: User \"system:node:ip-10-0-133-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-207.ec2.internal' and this object" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:19.682498 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.682464 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume\") pod \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " Jul 06 19:32:19.682664 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.682528 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shr4t\" (UniqueName: \"kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t\") pod \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\" (UID: \"24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f\") " Jul 06 19:32:19.682733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.682711 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" (UID: "24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:19.684678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.684659 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t" (OuterVolumeSpecName: "kube-api-access-shr4t") pod "24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" (UID: "24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f"). InnerVolumeSpecName "kube-api-access-shr4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:19.784005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.783938 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-extensions-socket-volume\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:32:19.784005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:19.783960 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shr4t\" (UniqueName: \"kubernetes.io/projected/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f-kube-api-access-shr4t\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:32:20.280094 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.280058 2555 generic.go:356] "Generic (PLEG): container finished" podID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" containerID="6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e" exitCode=2 Jul 06 19:32:20.280554 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.280117 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:20.280554 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.280164 2555 scope.go:117] "RemoveContainer" containerID="6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e" Jul 06 19:32:20.282747 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:20.282719 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6s259\" is forbidden: User \"system:node:ip-10-0-133-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-207.ec2.internal' and this object" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:20.288842 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.288807 2555 scope.go:117] "RemoveContainer" containerID="6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e" Jul 06 19:32:20.289090 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:20.289067 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e\": container with ID starting with 6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e not found: ID does not exist" containerID="6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e" Jul 06 19:32:20.289141 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.289101 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e"} err="failed to get container status \"6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e\": rpc error: code = NotFound desc = could not find container \"6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e\": container with ID starting with 6004244f08c8859dd111336cafdcc82b5febbaa28c740582c6a67da85278f11e not found: ID does not exist" Jul 06 19:32:20.290832 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:20.290812 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6s259\" is forbidden: User \"system:node:ip-10-0-133-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-207.ec2.internal' and this object" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6s259" Jul 06 19:32:20.967020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:20.966986 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" path="/var/lib/kubelet/pods/24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f/volumes" Jul 06 19:32:21.749037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:21.749010 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:21.850685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:21.850654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:28.235980 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:28.235943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:28.294632 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:28.294578 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:30.282300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:30.282216 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:33.006809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:33.006775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:33.114278 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:33.114243 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:36.991790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:36.991754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.068476 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:37.068448 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.597007 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:37.596977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.656058 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:37.656030 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:41.188121 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:41.188081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:41.261894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:41.261865 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:42.766168 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:42.766136 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:42.766544 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:42.766362 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" podUID="f75059e8-7c9b-41a4-908f-326fd3b292c6" containerName="manager" containerID="cri-o://427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372" gracePeriod=10 Jul 06 19:32:43.018470 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.018417 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:43.079998 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.079970 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume\") pod \"f75059e8-7c9b-41a4-908f-326fd3b292c6\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " Jul 06 19:32:43.080159 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.080033 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kzcl\" (UniqueName: \"kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl\") pod \"f75059e8-7c9b-41a4-908f-326fd3b292c6\" (UID: \"f75059e8-7c9b-41a4-908f-326fd3b292c6\") " Jul 06 19:32:43.080437 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.080405 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f75059e8-7c9b-41a4-908f-326fd3b292c6" (UID: "f75059e8-7c9b-41a4-908f-326fd3b292c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:43.082234 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.082208 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl" (OuterVolumeSpecName: "kube-api-access-6kzcl") pod "f75059e8-7c9b-41a4-908f-326fd3b292c6" (UID: "f75059e8-7c9b-41a4-908f-326fd3b292c6"). InnerVolumeSpecName "kube-api-access-6kzcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:43.181250 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.181220 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75059e8-7c9b-41a4-908f-326fd3b292c6-extensions-socket-volume\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:32:43.181250 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.181245 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kzcl\" (UniqueName: \"kubernetes.io/projected/f75059e8-7c9b-41a4-908f-326fd3b292c6-kube-api-access-6kzcl\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:32:43.389065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.388982 2555 generic.go:356] "Generic (PLEG): container finished" podID="f75059e8-7c9b-41a4-908f-326fd3b292c6" containerID="427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372" exitCode=0 Jul 06 19:32:43.389065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.389031 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" Jul 06 19:32:43.389227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.389062 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" event={"ID":"f75059e8-7c9b-41a4-908f-326fd3b292c6","Type":"ContainerDied","Data":"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372"} Jul 06 19:32:43.389227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.389115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f" event={"ID":"f75059e8-7c9b-41a4-908f-326fd3b292c6","Type":"ContainerDied","Data":"32ab8d137ddeb68015d61fbc5e4ac61eab718264150aac93728a2055ce725707"} Jul 06 19:32:43.389227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.389137 2555 scope.go:117] "RemoveContainer" containerID="427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372" Jul 06 19:32:43.398410 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.398392 2555 scope.go:117] "RemoveContainer" containerID="427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372" Jul 06 19:32:43.398669 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:32:43.398649 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372\": container with ID starting with 427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372 not found: ID does not exist" containerID="427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372" Jul 06 19:32:43.398741 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.398681 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372"} err="failed to get container status \"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372\": rpc error: code = NotFound desc = could not find container \"427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372\": container with ID starting with 427a1e3bcc1460a9eedc82558f4157f259d9e97052a845f8607ae8d6b00dd372 not found: ID does not exist" Jul 06 19:32:43.413075 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.413054 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:43.419079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:43.419060 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fm82f"] Jul 06 19:32:44.968655 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:44.968627 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f75059e8-7c9b-41a4-908f-326fd3b292c6" path="/var/lib/kubelet/pods/f75059e8-7c9b-41a4-908f-326fd3b292c6/volumes" Jul 06 19:32:45.178489 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:45.178459 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:46.700049 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:46.700018 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.110134 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:47.110042 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.323400 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:47.323364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.393695 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:47.393607 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:52.020068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:52.020029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:52.118130 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:52.118091 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:54.590296 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590266 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c4f849554-v8nlx"] Jul 06 19:32:54.590698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590605 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" containerName="manager" Jul 06 19:32:54.590698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590621 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" containerName="manager" Jul 06 19:32:54.590698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590632 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f75059e8-7c9b-41a4-908f-326fd3b292c6" containerName="manager" Jul 06 19:32:54.590698 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590637 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f75059e8-7c9b-41a4-908f-326fd3b292c6" containerName="manager" Jul 06 19:32:54.590849 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590709 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d06ce7-6ac2-4a0c-8d72-b238fa60ab7f" containerName="manager" Jul 06 19:32:54.590849 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.590719 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f75059e8-7c9b-41a4-908f-326fd3b292c6" containerName="manager" Jul 06 19:32:54.596548 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.596527 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.607391 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.607369 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4f849554-v8nlx"] Jul 06 19:32:54.671645 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.671565 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:54.682745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682713 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-service-ca\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.682859 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8p4\" (UniqueName: \"kubernetes.io/projected/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-kube-api-access-7j8p4\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.682859 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682793 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.682859 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682827 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-oauth-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.682859 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682848 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-oauth-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.683040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682896 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-trusted-ca-bundle\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.683040 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.682941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.749654 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.749618 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:54.784315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-oauth-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784399 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784321 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-oauth-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784399 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784343 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-trusted-ca-bundle\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784399 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784386 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784531 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-service-ca\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784658 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8p4\" (UniqueName: \"kubernetes.io/projected/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-kube-api-access-7j8p4\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.784746 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.784699 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.785474 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.785445 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-oauth-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.785673 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.785495 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-service-ca\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.785896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.785875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.787918 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.785981 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-trusted-ca-bundle\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.790896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.788039 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-serving-cert\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.790896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.788438 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-console-oauth-config\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.801789 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.801772 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8p4\" (UniqueName: \"kubernetes.io/projected/90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4-kube-api-access-7j8p4\") pod \"console-7c4f849554-v8nlx\" (UID: \"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4\") " pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:54.906625 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:54.906547 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:32:55.044627 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:55.044587 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4f849554-v8nlx"] Jul 06 19:32:55.047022 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:32:55.046998 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c3d902_6cd5_4767_9cb6_83cd1c7cc8b4.slice/crio-0eb16fdfac2c863a3f2ef23963e25810e9a17e424970749f4b997b69e0e8c15c WatchSource:0}: Error finding container 0eb16fdfac2c863a3f2ef23963e25810e9a17e424970749f4b997b69e0e8c15c: Status 404 returned error can't find the container with id 0eb16fdfac2c863a3f2ef23963e25810e9a17e424970749f4b997b69e0e8c15c Jul 06 19:32:55.445525 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:55.445491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f849554-v8nlx" event={"ID":"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4","Type":"ContainerStarted","Data":"6c47ed1ff1901ba7a11bbcad763854f15df9a6a5718d3c54ea8d8aec84163ece"} Jul 06 19:32:55.445702 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:55.445541 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f849554-v8nlx" event={"ID":"90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4","Type":"ContainerStarted","Data":"0eb16fdfac2c863a3f2ef23963e25810e9a17e424970749f4b997b69e0e8c15c"} Jul 06 19:32:55.478680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:55.478620 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c4f849554-v8nlx" podStartSLOduration=1.478606228 podStartE2EDuration="1.478606228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:55.477056367 +0000 UTC m=+556.982654851" watchObservedRunningTime="2026-07-06 19:32:55.478606228 +0000 UTC m=+556.984204701" Jul 06 19:32:59.569559 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.569526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.573829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.573780 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:32:59.581646 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.581623 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.584743 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.584722 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 19:32:59.585373 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.585353 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:32:59.585472 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.585353 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:32:59.585472 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.585359 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vpt7f\"" Jul 06 19:32:59.586059 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.586032 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:32:59.681627 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.681581 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:32:59.682352 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.682322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.733911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.733884 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x69ps\" (UniqueName: \"kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.734041 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.733925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.781903 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.781869 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.835229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.835166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.835342 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.835268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x69ps\" (UniqueName: \"kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.835802 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.835784 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.845888 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.845869 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x69ps\" (UniqueName: \"kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps\") pod \"limitador-limitador-64c94fd64f-9b4jw\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:32:59.893754 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:32:59.893732 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:33:00.026338 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:00.026312 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:33:00.028324 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:33:00.028295 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc556a4ec_74a0_47fe_b157_34b7ca14e314.slice/crio-2f9fcfc36ba931546087edaa2b368b237e7656eb2a5611bdb4cf759ccde29033 WatchSource:0}: Error finding container 2f9fcfc36ba931546087edaa2b368b237e7656eb2a5611bdb4cf759ccde29033: Status 404 returned error can't find the container with id 2f9fcfc36ba931546087edaa2b368b237e7656eb2a5611bdb4cf759ccde29033 Jul 06 19:33:00.468720 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:00.468664 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" event={"ID":"c556a4ec-74a0-47fe-b157-34b7ca14e314","Type":"ContainerStarted","Data":"2f9fcfc36ba931546087edaa2b368b237e7656eb2a5611bdb4cf759ccde29033"} Jul 06 19:33:00.490548 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:00.490513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:00.580789 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:00.580760 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:03.484495 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:03.484462 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" event={"ID":"c556a4ec-74a0-47fe-b157-34b7ca14e314","Type":"ContainerStarted","Data":"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496"} Jul 06 19:33:03.484875 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:03.484533 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:33:03.506315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:03.506269 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" podStartSLOduration=1.8585579509999999 podStartE2EDuration="4.506255578s" totalImagesPullingTime="2.647697627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:03.505312466 +0000 UTC m=+565.010910949" watchObservedRunningTime="2026-07-06 19:33:03.506255578 +0000 UTC m=+565.011854060" Jul 06 19:33:04.907124 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:04.907089 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:33:04.907570 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:04.907147 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:33:04.911761 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:04.911739 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:33:05.497312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:05.497284 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c4f849554-v8nlx" Jul 06 19:33:05.563794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:05.563752 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:33:05.667947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:05.667917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:05.768813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:05.768728 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:05.868461 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:05.868431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:14.490744 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:14.490707 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:33:16.831032 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:16.830998 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:33:16.831407 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:16.831248 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" podUID="c556a4ec-74a0-47fe-b157-34b7ca14e314" containerName="limitador" containerID="cri-o://5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496" gracePeriod=30 Jul 06 19:33:17.381756 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.381733 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:33:17.486497 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.486463 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file\") pod \"c556a4ec-74a0-47fe-b157-34b7ca14e314\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " Jul 06 19:33:17.486694 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.486512 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x69ps\" (UniqueName: \"kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps\") pod \"c556a4ec-74a0-47fe-b157-34b7ca14e314\" (UID: \"c556a4ec-74a0-47fe-b157-34b7ca14e314\") " Jul 06 19:33:17.486876 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.486855 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file" (OuterVolumeSpecName: "config-file") pod "c556a4ec-74a0-47fe-b157-34b7ca14e314" (UID: "c556a4ec-74a0-47fe-b157-34b7ca14e314"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:17.488807 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.488782 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps" (OuterVolumeSpecName: "kube-api-access-x69ps") pod "c556a4ec-74a0-47fe-b157-34b7ca14e314" (UID: "c556a4ec-74a0-47fe-b157-34b7ca14e314"). InnerVolumeSpecName "kube-api-access-x69ps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:33:17.550097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.550057 2555 generic.go:356] "Generic (PLEG): container finished" podID="c556a4ec-74a0-47fe-b157-34b7ca14e314" containerID="5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496" exitCode=0 Jul 06 19:33:17.550226 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.550106 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" Jul 06 19:33:17.550226 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.550148 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" event={"ID":"c556a4ec-74a0-47fe-b157-34b7ca14e314","Type":"ContainerDied","Data":"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496"} Jul 06 19:33:17.550226 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.550197 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9b4jw" event={"ID":"c556a4ec-74a0-47fe-b157-34b7ca14e314","Type":"ContainerDied","Data":"2f9fcfc36ba931546087edaa2b368b237e7656eb2a5611bdb4cf759ccde29033"} Jul 06 19:33:17.550226 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.550217 2555 scope.go:117] "RemoveContainer" containerID="5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496" Jul 06 19:33:17.558883 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.558870 2555 scope.go:117] "RemoveContainer" containerID="5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496" Jul 06 19:33:17.559116 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:33:17.559095 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496\": container with ID starting with 5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496 not found: ID does not exist" containerID="5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496" Jul 06 19:33:17.559208 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.559120 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496"} err="failed to get container status \"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496\": rpc error: code = NotFound desc = could not find container \"5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496\": container with ID starting with 5c63ffb9ef7bc1a817c9bca9121e1e658a36dbcadec918edb18e47138bb26496 not found: ID does not exist" Jul 06 19:33:17.572873 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.572850 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:33:17.578943 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.578923 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9b4jw"] Jul 06 19:33:17.587513 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.587494 2555 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c556a4ec-74a0-47fe-b157-34b7ca14e314-config-file\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:17.587513 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:17.587514 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x69ps\" (UniqueName: \"kubernetes.io/projected/c556a4ec-74a0-47fe-b157-34b7ca14e314-kube-api-access-x69ps\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:18.967310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:18.967278 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c556a4ec-74a0-47fe-b157-34b7ca14e314" path="/var/lib/kubelet/pods/c556a4ec-74a0-47fe-b157-34b7ca14e314/volumes" Jul 06 19:33:21.517062 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.517033 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-cxrc9"] Jul 06 19:33:21.517423 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.517373 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c556a4ec-74a0-47fe-b157-34b7ca14e314" containerName="limitador" Jul 06 19:33:21.517423 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.517384 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c556a4ec-74a0-47fe-b157-34b7ca14e314" containerName="limitador" Jul 06 19:33:21.517510 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.517457 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c556a4ec-74a0-47fe-b157-34b7ca14e314" containerName="limitador" Jul 06 19:33:21.522108 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.522080 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.529556 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.529529 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 19:33:21.529921 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.529894 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mt6bf\"" Jul 06 19:33:21.534366 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.534339 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cxrc9"] Jul 06 19:33:21.553979 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.553957 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:21.621482 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.621451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb9e0057-0bea-4a48-842d-9eae25716f12-data\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.621651 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.621578 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvcd9\" (UniqueName: \"kubernetes.io/projected/eb9e0057-0bea-4a48-842d-9eae25716f12-kube-api-access-bvcd9\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.722691 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.722658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvcd9\" (UniqueName: \"kubernetes.io/projected/eb9e0057-0bea-4a48-842d-9eae25716f12-kube-api-access-bvcd9\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.722883 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.722711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb9e0057-0bea-4a48-842d-9eae25716f12-data\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.723053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.723032 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb9e0057-0bea-4a48-842d-9eae25716f12-data\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.734102 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.734072 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvcd9\" (UniqueName: \"kubernetes.io/projected/eb9e0057-0bea-4a48-842d-9eae25716f12-kube-api-access-bvcd9\") pod \"postgres-69fb4594cf-cxrc9\" (UID: \"eb9e0057-0bea-4a48-842d-9eae25716f12\") " pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.840296 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.840222 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:21.969038 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:21.968963 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cxrc9"] Jul 06 19:33:21.971835 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:33:21.971809 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9e0057_0bea_4a48_842d_9eae25716f12.slice/crio-89078eb4ab5784ca93c055c7acc95332ff3986fa2dfe22260d669c3558ced08c WatchSource:0}: Error finding container 89078eb4ab5784ca93c055c7acc95332ff3986fa2dfe22260d669c3558ced08c: Status 404 returned error can't find the container with id 89078eb4ab5784ca93c055c7acc95332ff3986fa2dfe22260d669c3558ced08c Jul 06 19:33:22.575951 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:22.575912 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cxrc9" event={"ID":"eb9e0057-0bea-4a48-842d-9eae25716f12","Type":"ContainerStarted","Data":"89078eb4ab5784ca93c055c7acc95332ff3986fa2dfe22260d669c3558ced08c"} Jul 06 19:33:27.604285 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:27.604252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cxrc9" event={"ID":"eb9e0057-0bea-4a48-842d-9eae25716f12","Type":"ContainerStarted","Data":"a056384258f2de58d34011b0c11fd1a2e1713fb29f209b74899f3a0132393bb6"} Jul 06 19:33:27.604776 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:27.604316 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:27.623813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:27.623764 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-cxrc9" podStartSLOduration=1.9380200159999998 podStartE2EDuration="6.623748451s" totalImagesPullingTime="4.685728435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:27.623437941 +0000 UTC m=+589.129036425" watchObservedRunningTime="2026-07-06 19:33:27.623748451 +0000 UTC m=+589.129346932" Jul 06 19:33:30.591500 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.591459 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d755c5c45-54vxb" podUID="6d778533-a4db-4747-b796-a2ae29e5694f" containerName="console" containerID="cri-o://4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70" gracePeriod=15 Jul 06 19:33:30.841787 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.841730 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:33:30.909954 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.909926 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910140 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.909970 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910140 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.909998 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2wws\" (UniqueName: \"kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910140 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910114 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910146 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910199 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910218 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle\") pod \"6d778533-a4db-4747-b796-a2ae29e5694f\" (UID: \"6d778533-a4db-4747-b796-a2ae29e5694f\") " Jul 06 19:33:30.910527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910430 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:30.910527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910399 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config" (OuterVolumeSpecName: "console-config") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:30.910662 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910618 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-console-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:30.910662 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910640 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-oauth-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:30.910755 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910562 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:30.910795 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.910757 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:30.912549 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.912524 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:33:30.912878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.912852 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:33:30.912952 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:30.912859 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws" (OuterVolumeSpecName: "kube-api-access-l2wws") pod "6d778533-a4db-4747-b796-a2ae29e5694f" (UID: "6d778533-a4db-4747-b796-a2ae29e5694f"). InnerVolumeSpecName "kube-api-access-l2wws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:33:31.011310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.011275 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-serving-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:31.011310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.011303 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d778533-a4db-4747-b796-a2ae29e5694f-console-oauth-config\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:31.011310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.011313 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:31.011557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.011325 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d778533-a4db-4747-b796-a2ae29e5694f-trusted-ca-bundle\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:31.011557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.011340 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2wws\" (UniqueName: \"kubernetes.io/projected/6d778533-a4db-4747-b796-a2ae29e5694f-kube-api-access-l2wws\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:31.623865 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.623827 2555 generic.go:356] "Generic (PLEG): container finished" podID="6d778533-a4db-4747-b796-a2ae29e5694f" containerID="4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70" exitCode=0 Jul 06 19:33:31.623865 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.623867 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d755c5c45-54vxb" event={"ID":"6d778533-a4db-4747-b796-a2ae29e5694f","Type":"ContainerDied","Data":"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70"} Jul 06 19:33:31.624370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.623884 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d755c5c45-54vxb" Jul 06 19:33:31.624370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.623907 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d755c5c45-54vxb" event={"ID":"6d778533-a4db-4747-b796-a2ae29e5694f","Type":"ContainerDied","Data":"9b4238527a6b1bc6e19e34225708a905c8c52d883f19cdf2b4562b5757035762"} Jul 06 19:33:31.624370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.623925 2555 scope.go:117] "RemoveContainer" containerID="4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70" Jul 06 19:33:31.632428 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.632412 2555 scope.go:117] "RemoveContainer" containerID="4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70" Jul 06 19:33:31.632721 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:33:31.632698 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70\": container with ID starting with 4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70 not found: ID does not exist" containerID="4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70" Jul 06 19:33:31.632814 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.632727 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70"} err="failed to get container status \"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70\": rpc error: code = NotFound desc = could not find container \"4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70\": container with ID starting with 4e599da7d6bb7892923b2813747d78d00fc943b7baeb235c0b09ffdbda301e70 not found: ID does not exist" Jul 06 19:33:31.642674 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.642648 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:33:31.646426 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:31.646404 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d755c5c45-54vxb"] Jul 06 19:33:32.967641 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:32.967583 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d778533-a4db-4747-b796-a2ae29e5694f" path="/var/lib/kubelet/pods/6d778533-a4db-4747-b796-a2ae29e5694f/volumes" Jul 06 19:33:33.639473 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:33.639439 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-cxrc9" Jul 06 19:33:33.974167 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:33.974137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:34.563955 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:34.563926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.555232 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.555199 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.561280 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.561255 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:40.561623 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.561610 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d778533-a4db-4747-b796-a2ae29e5694f" containerName="console" Jul 06 19:33:40.561668 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.561625 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d778533-a4db-4747-b796-a2ae29e5694f" containerName="console" Jul 06 19:33:40.561700 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.561691 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d778533-a4db-4747-b796-a2ae29e5694f" containerName="console" Jul 06 19:33:40.571192 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.571163 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.572651 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.572626 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:40.574098 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.574075 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:33:40.574326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.574239 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tlz9n\"" Jul 06 19:33:40.574326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.574078 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:33:40.659824 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.659793 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.692853 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.692825 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.692996 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.692872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm9kw\" (UniqueName: \"kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.732738 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.732699 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:33:40.737716 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.737681 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:40.744668 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.744642 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:33:40.745861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.745837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.793922 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.793886 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.794071 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.793943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm9kw\" (UniqueName: \"kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.794071 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:33:40.794046 2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 19:33:40.794206 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:33:40.794123 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert podName:322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b nodeName:}" failed. No retries permitted until 2026-07-06 19:33:41.29409585 +0000 UTC m=+602.799694324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert") pod "maas-controller-6ddd4c6dfb-25wgm" (UID: "322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b") : secret "maas-controller-webhook-cert" not found Jul 06 19:33:40.803465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.803432 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm9kw\" (UniqueName: \"kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:40.895008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.894922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntjh\" (UniqueName: \"kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:40.895008 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.894969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:40.971451 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.971419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.995692 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.995665 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntjh\" (UniqueName: \"kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:40.995815 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.995705 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:40.998149 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:40.998123 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:41.005682 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.005655 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntjh\" (UniqueName: \"kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh\") pod \"maas-controller-746f49cdbd-f5b8r\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:41.051367 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.051342 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:41.185510 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.185484 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:33:41.187437 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:33:41.187412 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5a4c0f_818a_4517_b35a_d40c3997a31d.slice/crio-6fd49f9ddcfb013ed5ce91835e51e0c7294b82712c9741e15f29de7b7214b71c WatchSource:0}: Error finding container 6fd49f9ddcfb013ed5ce91835e51e0c7294b82712c9741e15f29de7b7214b71c: Status 404 returned error can't find the container with id 6fd49f9ddcfb013ed5ce91835e51e0c7294b82712c9741e15f29de7b7214b71c Jul 06 19:33:41.299099 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.299065 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:41.301389 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.301367 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") pod \"maas-controller-6ddd4c6dfb-25wgm\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:41.483578 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.483546 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:41.612488 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.612463 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:41.614374 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:33:41.614347 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322cb3d6_e52c_4c6b_a2d0_e0c22bb8969b.slice/crio-a00f6cbe4eb4f3d3bc5e62021a2543a6a8b688b6b2ea422a27fa25db5b7d80fb WatchSource:0}: Error finding container a00f6cbe4eb4f3d3bc5e62021a2543a6a8b688b6b2ea422a27fa25db5b7d80fb: Status 404 returned error can't find the container with id a00f6cbe4eb4f3d3bc5e62021a2543a6a8b688b6b2ea422a27fa25db5b7d80fb Jul 06 19:33:41.672500 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.672471 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" event={"ID":"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b","Type":"ContainerStarted","Data":"a00f6cbe4eb4f3d3bc5e62021a2543a6a8b688b6b2ea422a27fa25db5b7d80fb"} Jul 06 19:33:41.673482 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:41.673460 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerStarted","Data":"6fd49f9ddcfb013ed5ce91835e51e0c7294b82712c9741e15f29de7b7214b71c"} Jul 06 19:33:44.284392 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.284353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:44.376794 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.376704 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:44.692065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.691958 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" event={"ID":"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b","Type":"ContainerStarted","Data":"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863"} Jul 06 19:33:44.692065 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.692022 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:44.693507 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.693480 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerStarted","Data":"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052"} Jul 06 19:33:44.693643 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.693529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:44.709846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.709807 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" podStartSLOduration=2.223032149 podStartE2EDuration="4.709792938s" totalImagesPullingTime="2.486760789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:44.709168656 +0000 UTC m=+606.214767138" watchObservedRunningTime="2026-07-06 19:33:44.709792938 +0000 UTC m=+606.215391419" Jul 06 19:33:44.725938 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:44.725893 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" podStartSLOduration=1.822487627 podStartE2EDuration="4.725879302s" totalImagesPullingTime="2.903391675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:44.725164544 +0000 UTC m=+606.230763039" watchObservedRunningTime="2026-07-06 19:33:44.725879302 +0000 UTC m=+606.231477784" Jul 06 19:33:55.704255 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:55.704221 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:33:55.704790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:55.704302 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:55.748812 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:55.748785 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:55.748989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:55.748962 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" podUID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" containerName="manager" containerID="cri-o://9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863" gracePeriod=30 Jul 06 19:33:55.998708 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:55.998679 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:56.133241 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.133209 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") pod \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " Jul 06 19:33:56.133403 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.133283 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm9kw\" (UniqueName: \"kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw\") pod \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\" (UID: \"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b\") " Jul 06 19:33:56.135785 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.135745 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw" (OuterVolumeSpecName: "kube-api-access-zm9kw") pod "322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" (UID: "322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b"). InnerVolumeSpecName "kube-api-access-zm9kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:33:56.135785 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.135764 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" (UID: "322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:33:56.234050 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.233967 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-webhook-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:56.234050 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.233993 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm9kw\" (UniqueName: \"kubernetes.io/projected/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b-kube-api-access-zm9kw\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:33:56.750922 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.750883 2555 generic.go:356] "Generic (PLEG): container finished" podID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" containerID="9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863" exitCode=0 Jul 06 19:33:56.751326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.750930 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" Jul 06 19:33:56.751326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.750965 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" event={"ID":"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b","Type":"ContainerDied","Data":"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863"} Jul 06 19:33:56.751326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.751033 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6ddd4c6dfb-25wgm" event={"ID":"322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b","Type":"ContainerDied","Data":"a00f6cbe4eb4f3d3bc5e62021a2543a6a8b688b6b2ea422a27fa25db5b7d80fb"} Jul 06 19:33:56.751326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.751056 2555 scope.go:117] "RemoveContainer" containerID="9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863" Jul 06 19:33:56.760148 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.760133 2555 scope.go:117] "RemoveContainer" containerID="9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863" Jul 06 19:33:56.760423 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:33:56.760403 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863\": container with ID starting with 9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863 not found: ID does not exist" containerID="9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863" Jul 06 19:33:56.760510 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.760434 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863"} err="failed to get container status \"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863\": rpc error: code = NotFound desc = could not find container \"9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863\": container with ID starting with 9a04c5f76760c3b8e09f9ae6586fdb97f99534176bd1b8f0a8117a300bcbe863 not found: ID does not exist" Jul 06 19:33:56.774527 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.774503 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:56.778747 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.778721 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6ddd4c6dfb-25wgm"] Jul 06 19:33:56.967259 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:33:56.967229 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" path="/var/lib/kubelet/pods/322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b/volumes" Jul 06 19:34:14.884641 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:14.884534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:15.451542 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.451508 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:15.508509 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.508479 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-76c64f7c7b-bp2k4"] Jul 06 19:34:15.508894 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.508880 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" containerName="manager" Jul 06 19:34:15.508955 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.508898 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" containerName="manager" Jul 06 19:34:15.508991 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.508970 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="322cb3d6-e52c-4c6b-a2d0-e0c22bb8969b" containerName="manager" Jul 06 19:34:15.511947 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.511932 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.514850 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.514829 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:34:15.515048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.515031 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jplkb\"" Jul 06 19:34:15.521757 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.521734 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76c64f7c7b-bp2k4"] Jul 06 19:34:15.588654 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.588618 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:15.593765 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.593748 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75a13851-17da-4dad-9174-d48f18ca1ebc-maas-api-tls\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.593847 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.593772 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a13851-17da-4dad-9174-d48f18ca1ebc-openshift-service-ca\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.593847 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.593815 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv4v4\" (UniqueName: \"kubernetes.io/projected/75a13851-17da-4dad-9174-d48f18ca1ebc-kube-api-access-zv4v4\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.695122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.695084 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4v4\" (UniqueName: \"kubernetes.io/projected/75a13851-17da-4dad-9174-d48f18ca1ebc-kube-api-access-zv4v4\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.695291 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.695156 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75a13851-17da-4dad-9174-d48f18ca1ebc-maas-api-tls\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.695291 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.695175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a13851-17da-4dad-9174-d48f18ca1ebc-openshift-service-ca\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.695798 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.695776 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a13851-17da-4dad-9174-d48f18ca1ebc-openshift-service-ca\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.697678 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.697656 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75a13851-17da-4dad-9174-d48f18ca1ebc-maas-api-tls\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.704616 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.704534 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv4v4\" (UniqueName: \"kubernetes.io/projected/75a13851-17da-4dad-9174-d48f18ca1ebc-kube-api-access-zv4v4\") pod \"maas-api-76c64f7c7b-bp2k4\" (UID: \"75a13851-17da-4dad-9174-d48f18ca1ebc\") " pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.824776 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.824743 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:15.913617 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.913060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:15.999674 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:15.999621 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76c64f7c7b-bp2k4"] Jul 06 19:34:16.001995 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:34:16.001967 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a13851_17da_4dad_9174_d48f18ca1ebc.slice/crio-c73ae0c78d5f426b4749924c637bff7ba9b37f4ecaefef63040c5e51859ff4ad WatchSource:0}: Error finding container c73ae0c78d5f426b4749924c637bff7ba9b37f4ecaefef63040c5e51859ff4ad: Status 404 returned error can't find the container with id c73ae0c78d5f426b4749924c637bff7ba9b37f4ecaefef63040c5e51859ff4ad Jul 06 19:34:16.002930 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.002903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:16.040532 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.040498 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mx8xg"] Jul 06 19:34:16.045020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.044997 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.048041 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.048020 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:34:16.048215 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.048078 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w6d9f\"" Jul 06 19:34:16.051072 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.051049 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dwns9"] Jul 06 19:34:16.055067 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.055048 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mx8xg"] Jul 06 19:34:16.055171 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.055133 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.065486 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.065465 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dwns9"] Jul 06 19:34:16.088154 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.088115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:16.107147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.107119 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.107273 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.107165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbgkg\" (UniqueName: \"kubernetes.io/projected/a3d6c60b-a840-4b99-b709-aef3dba23b81-kube-api-access-jbgkg\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.107337 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.107269 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d6c60b-a840-4b99-b709-aef3dba23b81-plugins-config-volume\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.107337 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.107314 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9v89\" (UniqueName: \"kubernetes.io/projected/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-kube-api-access-c9v89\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.208715 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.208679 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgkg\" (UniqueName: \"kubernetes.io/projected/a3d6c60b-a840-4b99-b709-aef3dba23b81-kube-api-access-jbgkg\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.208878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.208742 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d6c60b-a840-4b99-b709-aef3dba23b81-plugins-config-volume\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.208878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.208777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9v89\" (UniqueName: \"kubernetes.io/projected/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-kube-api-access-c9v89\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.208878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.208846 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.209470 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.209449 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d6c60b-a840-4b99-b709-aef3dba23b81-plugins-config-volume\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.209537 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.209489 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.218238 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.218207 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9v89\" (UniqueName: \"kubernetes.io/projected/6e0a1082-94ab-48fd-bc75-b77e871f4f2c-kube-api-access-c9v89\") pod \"payload-pre-processing-7964457696-mx8xg\" (UID: \"6e0a1082-94ab-48fd-bc75-b77e871f4f2c\") " pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.218475 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.218459 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbgkg\" (UniqueName: \"kubernetes.io/projected/a3d6c60b-a840-4b99-b709-aef3dba23b81-kube-api-access-jbgkg\") pod \"payload-processing-58f86c7798-dwns9\" (UID: \"a3d6c60b-a840-4b99-b709-aef3dba23b81\") " pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.358183 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.358090 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:16.368841 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.368817 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:16.505266 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.505231 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mx8xg"] Jul 06 19:34:16.506874 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:34:16.506846 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0a1082_94ab_48fd_bc75_b77e871f4f2c.slice/crio-7b2467e8d4d47ea0ed217c8c4b83c86436fc2e172dd4cd7a49dfef12d7f280f4 WatchSource:0}: Error finding container 7b2467e8d4d47ea0ed217c8c4b83c86436fc2e172dd4cd7a49dfef12d7f280f4: Status 404 returned error can't find the container with id 7b2467e8d4d47ea0ed217c8c4b83c86436fc2e172dd4cd7a49dfef12d7f280f4 Jul 06 19:34:16.524569 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.524441 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dwns9"] Jul 06 19:34:16.843397 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.843349 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" event={"ID":"a3d6c60b-a840-4b99-b709-aef3dba23b81","Type":"ContainerStarted","Data":"2a49ac93f21c3dd87e3aac71f4546474468a4f9610ce3e4537e1021e2868f0c0"} Jul 06 19:34:16.844642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.844614 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" event={"ID":"75a13851-17da-4dad-9174-d48f18ca1ebc","Type":"ContainerStarted","Data":"c73ae0c78d5f426b4749924c637bff7ba9b37f4ecaefef63040c5e51859ff4ad"} Jul 06 19:34:16.845657 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:16.845639 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" event={"ID":"6e0a1082-94ab-48fd-bc75-b77e871f4f2c","Type":"ContainerStarted","Data":"7b2467e8d4d47ea0ed217c8c4b83c86436fc2e172dd4cd7a49dfef12d7f280f4"} Jul 06 19:34:18.859983 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.859947 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" event={"ID":"75a13851-17da-4dad-9174-d48f18ca1ebc","Type":"ContainerStarted","Data":"f123323e9b6301b73eb238ff34d590c748899abe5e846f528f0fdf0801d29afa"} Jul 06 19:34:18.860423 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.860116 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:18.861389 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.861360 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" event={"ID":"6e0a1082-94ab-48fd-bc75-b77e871f4f2c","Type":"ContainerStarted","Data":"be0fcdd8198b0121cb0a6cb61369fb476280ca0b3ac51ef8a9c1ecab835ac85a"} Jul 06 19:34:18.861540 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.861434 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:18.862718 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.862700 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" event={"ID":"a3d6c60b-a840-4b99-b709-aef3dba23b81","Type":"ContainerStarted","Data":"7d371ff66f166f259927644d4b025541d31c49562ebd0bc323ddf692e89180d9"} Jul 06 19:34:18.862829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.862812 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:18.877988 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.877929 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" podStartSLOduration=1.700190608 podStartE2EDuration="3.87791212s" totalImagesPullingTime="2.177721512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:18.877713115 +0000 UTC m=+640.383311599" watchObservedRunningTime="2026-07-06 19:34:18.87791212 +0000 UTC m=+640.383510606" Jul 06 19:34:18.895642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.895568 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" podStartSLOduration=1.242447123 podStartE2EDuration="2.895556165s" totalImagesPullingTime="1.653109042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:18.893884244 +0000 UTC m=+640.399482739" watchObservedRunningTime="2026-07-06 19:34:18.895556165 +0000 UTC m=+640.401154647" Jul 06 19:34:18.911060 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:18.911019 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" podStartSLOduration=1.239405854 podStartE2EDuration="2.911006927s" totalImagesPullingTime="1.671601073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:18.909313049 +0000 UTC m=+640.414911542" watchObservedRunningTime="2026-07-06 19:34:18.911006927 +0000 UTC m=+640.416605409" Jul 06 19:34:24.873613 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:24.873569 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-76c64f7c7b-bp2k4" Jul 06 19:34:25.843201 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:25.843159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:25.941957 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:25.941928 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:29.870552 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:29.870520 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-mx8xg" Jul 06 19:34:29.871020 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:29.870619 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-dwns9" Jul 06 19:34:36.434496 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.434455 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k"] Jul 06 19:34:36.438835 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.438813 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.441722 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.441699 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:34:36.443077 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.443055 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 19:34:36.443153 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.443091 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wxh4n\"" Jul 06 19:34:36.443153 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.443128 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:34:36.447432 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.447412 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k"] Jul 06 19:34:36.464166 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.464109 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:36.597570 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.597530 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.597866 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.597843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.598077 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.598042 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a361e235-5a1c-49f2-a00b-f7193add49d1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.598191 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.598098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.598191 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.598131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.598306 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.598224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8j25\" (UniqueName: \"kubernetes.io/projected/a361e235-5a1c-49f2-a00b-f7193add49d1-kube-api-access-h8j25\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.698681 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698647 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.698832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698692 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.698832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a361e235-5a1c-49f2-a00b-f7193add49d1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.698832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698765 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.698832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.699023 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.698915 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8j25\" (UniqueName: \"kubernetes.io/projected/a361e235-5a1c-49f2-a00b-f7193add49d1-kube-api-access-h8j25\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.699217 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.699193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.699329 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.699249 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.699329 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.699286 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.701181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.701160 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a361e235-5a1c-49f2-a00b-f7193add49d1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.701411 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.701394 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a361e235-5a1c-49f2-a00b-f7193add49d1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.707943 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.707922 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8j25\" (UniqueName: \"kubernetes.io/projected/a361e235-5a1c-49f2-a00b-f7193add49d1-kube-api-access-h8j25\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k\" (UID: \"a361e235-5a1c-49f2-a00b-f7193add49d1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.750410 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.750386 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:36.888934 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.888906 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k"] Jul 06 19:34:36.890847 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:34:36.890820 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda361e235_5a1c_49f2_a00b_f7193add49d1.slice/crio-b972e815b467f4968b2e2bedcd0c3780ce0124e318b2fd55ce50d6df36389d9f WatchSource:0}: Error finding container b972e815b467f4968b2e2bedcd0c3780ce0124e318b2fd55ce50d6df36389d9f: Status 404 returned error can't find the container with id b972e815b467f4968b2e2bedcd0c3780ce0124e318b2fd55ce50d6df36389d9f Jul 06 19:34:36.951454 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:36.951380 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" event={"ID":"a361e235-5a1c-49f2-a00b-f7193add49d1","Type":"ContainerStarted","Data":"b972e815b467f4968b2e2bedcd0c3780ce0124e318b2fd55ce50d6df36389d9f"} Jul 06 19:34:38.967417 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:38.967386 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" event={"ID":"a361e235-5a1c-49f2-a00b-f7193add49d1","Type":"ContainerStarted","Data":"5d83e6a2f840c71d4b089492fb3b0cc2dd903d84751ff0f557abe3da37f973b8"} Jul 06 19:34:38.985304 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:38.985259 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" podStartSLOduration=1.9463116710000001 podStartE2EDuration="2.985243816s" totalImagesPullingTime="1.038932145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:38.98338512 +0000 UTC m=+660.488983633" watchObservedRunningTime="2026-07-06 19:34:38.985243816 +0000 UTC m=+660.490842298" Jul 06 19:34:46.751324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:46.751286 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:46.751721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:46.751356 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:46.764006 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:46.763983 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:47.009746 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:47.009673 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k" Jul 06 19:34:48.017157 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:48.017128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:57.162724 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:34:57.162687 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:02.722446 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.722415 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8"] Jul 06 19:35:02.726155 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.726134 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.729083 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.729061 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 19:35:02.737294 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.737270 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8"] Jul 06 19:35:02.767978 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.767943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:02.847206 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/545c838a-8515-48b5-961b-5fddc038acea-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.847384 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.847384 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847291 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.847384 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.847384 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847366 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.847625 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.847483 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxqt\" (UniqueName: \"kubernetes.io/projected/545c838a-8515-48b5-961b-5fddc038acea-kube-api-access-gcxqt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948038 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948109 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948130 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948170 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxqt\" (UniqueName: \"kubernetes.io/projected/545c838a-8515-48b5-961b-5fddc038acea-kube-api-access-gcxqt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948212 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/545c838a-8515-48b5-961b-5fddc038acea-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948526 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948500 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948586 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948559 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.948663 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.948650 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.950662 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.950636 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/545c838a-8515-48b5-961b-5fddc038acea-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.950843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.950823 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/545c838a-8515-48b5-961b-5fddc038acea-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:02.957365 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:02.957341 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxqt\" (UniqueName: \"kubernetes.io/projected/545c838a-8515-48b5-961b-5fddc038acea-kube-api-access-gcxqt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5lsn8\" (UID: \"545c838a-8515-48b5-961b-5fddc038acea\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:03.037745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:03.037646 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:03.175199 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:03.175175 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8"] Jul 06 19:35:03.177077 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:35:03.177043 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545c838a_8515_48b5_961b_5fddc038acea.slice/crio-8a131fb8f4a03611bcba7995b982742a7d7d7b6e8b68592d6d8e9bae7ade2cd7 WatchSource:0}: Error finding container 8a131fb8f4a03611bcba7995b982742a7d7d7b6e8b68592d6d8e9bae7ade2cd7: Status 404 returned error can't find the container with id 8a131fb8f4a03611bcba7995b982742a7d7d7b6e8b68592d6d8e9bae7ade2cd7 Jul 06 19:35:03.178414 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:03.178398 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:35:04.078151 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:04.078112 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" event={"ID":"545c838a-8515-48b5-961b-5fddc038acea","Type":"ContainerStarted","Data":"409092b284425eb569e84347e4f1bd9df826a57e8d877bca7be18ea2c5acde37"} Jul 06 19:35:04.078690 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:04.078171 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" event={"ID":"545c838a-8515-48b5-961b-5fddc038acea","Type":"ContainerStarted","Data":"8a131fb8f4a03611bcba7995b982742a7d7d7b6e8b68592d6d8e9bae7ade2cd7"} Jul 06 19:35:04.121891 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:04.121826 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" podStartSLOduration=1.883188256 podStartE2EDuration="2.121803524s" totalImagesPullingTime="238.615268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:04.120378366 +0000 UTC m=+685.625976847" watchObservedRunningTime="2026-07-06 19:35:04.121803524 +0000 UTC m=+685.627402009" Jul 06 19:35:10.335305 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.335263 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm"] Jul 06 19:35:10.341829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.341812 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.344529 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.344508 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 19:35:10.348423 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.348394 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm"] Jul 06 19:35:10.418565 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.418530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:10.421942 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.421921 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.422015 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.421965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n24mt\" (UniqueName: \"kubernetes.io/projected/1c8b1c25-5c32-40b5-b697-649753a42c32-kube-api-access-n24mt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.422053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.422012 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c8b1c25-5c32-40b5-b697-649753a42c32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.422087 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.422059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.422087 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.422076 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.422162 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.422124 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.522680 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n24mt\" (UniqueName: \"kubernetes.io/projected/1c8b1c25-5c32-40b5-b697-649753a42c32-kube-api-access-n24mt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.522878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522694 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c8b1c25-5c32-40b5-b697-649753a42c32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.522878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522748 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.522878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.522878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522827 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.523162 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.522887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.523287 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.523262 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.523363 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.523290 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.523571 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.523534 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.525364 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.525333 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c8b1c25-5c32-40b5-b697-649753a42c32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.525501 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.525482 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c8b1c25-5c32-40b5-b697-649753a42c32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.533390 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.533362 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n24mt\" (UniqueName: \"kubernetes.io/projected/1c8b1c25-5c32-40b5-b697-649753a42c32-kube-api-access-n24mt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm\" (UID: \"1c8b1c25-5c32-40b5-b697-649753a42c32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.654340 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.654235 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:10.793800 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:10.793778 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm"] Jul 06 19:35:10.795872 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:35:10.795839 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8b1c25_5c32_40b5_b697_649753a42c32.slice/crio-fc10eb4a5dd4ad209fc5756e6dea3c261d2080b711cec8cfaafa79a49006a41c WatchSource:0}: Error finding container fc10eb4a5dd4ad209fc5756e6dea3c261d2080b711cec8cfaafa79a49006a41c: Status 404 returned error can't find the container with id fc10eb4a5dd4ad209fc5756e6dea3c261d2080b711cec8cfaafa79a49006a41c Jul 06 19:35:11.112402 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:11.112369 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" event={"ID":"1c8b1c25-5c32-40b5-b697-649753a42c32","Type":"ContainerStarted","Data":"849985daa8461f312f5e139bbcde2c9cac22c60742e93114fb2c4bba38cd372e"} Jul 06 19:35:11.112607 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:11.112429 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" event={"ID":"1c8b1c25-5c32-40b5-b697-649753a42c32","Type":"ContainerStarted","Data":"fc10eb4a5dd4ad209fc5756e6dea3c261d2080b711cec8cfaafa79a49006a41c"} Jul 06 19:35:11.135536 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:11.135486 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" podStartSLOduration=0.948184548 podStartE2EDuration="1.135469708s" totalImagesPullingTime="187.28516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:11.134524164 +0000 UTC m=+692.640122685" watchObservedRunningTime="2026-07-06 19:35:11.135469708 +0000 UTC m=+692.641068198" Jul 06 19:35:13.038832 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:13.038774 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:13.039282 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:13.038860 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:13.051710 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:13.051683 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:13.134193 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:13.134166 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5lsn8" Jul 06 19:35:20.654546 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:20.654510 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:20.654931 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:20.654575 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:20.667147 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:20.667123 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:21.173551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:21.173525 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm" Jul 06 19:35:22.316023 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:22.315990 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:28.975720 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:28.975680 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:37.756079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:37.756045 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:37.812809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:37.812780 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:40.193911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:35:40.193878 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:22.723382 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:22.723340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:23.432220 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:23.432184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:23.495325 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:23.495298 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:25.413047 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:25.413015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:25.512005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:25.511975 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:26.558303 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:26.558261 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:26.666107 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:26.666063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:58.188122 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:58.188080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:58.244675 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:36:58.244569 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:02.564238 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:02.564156 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:02.634889 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:02.634862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:26.700312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:26.700266 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.34:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:37:27.382151 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.382061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:27.462331 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.462295 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:27.765655 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.765625 2555 generic.go:356] "Generic (PLEG): container finished" podID="83bcb5f0-4604-4dcb-af4c-ef9b9f219685" containerID="dea4d65dd3058415ee83942af5cc65dd3204651b0aa95e9f64cefdad27051619" exitCode=1 Jul 06 19:37:27.766057 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.765697 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerDied","Data":"dea4d65dd3058415ee83942af5cc65dd3204651b0aa95e9f64cefdad27051619"} Jul 06 19:37:27.766195 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.766177 2555 scope.go:117] "RemoveContainer" containerID="dea4d65dd3058415ee83942af5cc65dd3204651b0aa95e9f64cefdad27051619" Jul 06 19:37:27.767407 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.767250 2555 patch_prober.go:28] interesting pod/lws-controller-manager-cf88bd784-wb2lg container/manager namespace/openshift-lws-operator: Readiness probe status=failure output="Get \"http://10.134.0.26:8081/readyz\": dial tcp 10.134.0.26:8081: connect: connection refused" start-of-body= Jul 06 19:37:27.767407 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.767268 2555 generic.go:356] "Generic (PLEG): container finished" podID="a29180a4-64ad-4de7-8080-8b974bed1a5f" containerID="d09b0e53d60c77958d34377dcc12678936c677b16b52ddccaf08cef566821fbf" exitCode=1 Jul 06 19:37:27.767407 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.767298 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" podUID="a29180a4-64ad-4de7-8080-8b974bed1a5f" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.26:8081/readyz\": dial tcp 10.134.0.26:8081: connect: connection refused" Jul 06 19:37:27.767407 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.767346 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" event={"ID":"a29180a4-64ad-4de7-8080-8b974bed1a5f","Type":"ContainerDied","Data":"d09b0e53d60c77958d34377dcc12678936c677b16b52ddccaf08cef566821fbf"} Jul 06 19:37:27.767693 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.767678 2555 scope.go:117] "RemoveContainer" containerID="d09b0e53d60c77958d34377dcc12678936c677b16b52ddccaf08cef566821fbf" Jul 06 19:37:27.769185 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.769073 2555 generic.go:356] "Generic (PLEG): container finished" podID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerID="46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052" exitCode=1 Jul 06 19:37:27.769185 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.769103 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerDied","Data":"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052"} Jul 06 19:37:27.769393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:27.769374 2555 scope.go:117] "RemoveContainer" containerID="46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052" Jul 06 19:37:28.776240 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.776210 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerStarted","Data":"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c"} Jul 06 19:37:28.776673 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.776410 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:37:28.778447 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.778426 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" event={"ID":"83bcb5f0-4604-4dcb-af4c-ef9b9f219685","Type":"ContainerStarted","Data":"096278aca2bb4efcaf8cc6dd0bc4dd9f0a2b18857c8e38d774b1edc04a9ddf44"} Jul 06 19:37:28.778725 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.778709 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:37:28.780186 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.780167 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" event={"ID":"a29180a4-64ad-4de7-8080-8b974bed1a5f","Type":"ContainerStarted","Data":"3c972a44feb69683fc17e12d1a663821b49012dd30d008a61e728ca905e08282"} Jul 06 19:37:28.780333 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:28.780320 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:37:39.787516 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:39.787488 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cf88bd784-wb2lg" Jul 06 19:37:39.788007 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:39.787583 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fcb8c7879-ztzgd" Jul 06 19:37:39.790687 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:37:39.790666 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:38:09.010084 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.010041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:09.121332 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.121300 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:38:09.121529 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.121506 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" containerID="cri-o://56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c" gracePeriod=30 Jul 06 19:38:09.374975 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.374950 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:38:09.515865 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.515827 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert\") pod \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " Jul 06 19:38:09.516050 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.515954 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pntjh\" (UniqueName: \"kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh\") pod \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\" (UID: \"ef5a4c0f-818a-4517-b35a-d40c3997a31d\") " Jul 06 19:38:09.518057 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.518029 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ef5a4c0f-818a-4517-b35a-d40c3997a31d" (UID: "ef5a4c0f-818a-4517-b35a-d40c3997a31d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:38:09.518057 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.518048 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh" (OuterVolumeSpecName: "kube-api-access-pntjh") pod "ef5a4c0f-818a-4517-b35a-d40c3997a31d" (UID: "ef5a4c0f-818a-4517-b35a-d40c3997a31d"). InnerVolumeSpecName "kube-api-access-pntjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:09.617784 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.617704 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5a4c0f-818a-4517-b35a-d40c3997a31d-webhook-cert\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:38:09.617784 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.617734 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pntjh\" (UniqueName: \"kubernetes.io/projected/ef5a4c0f-818a-4517-b35a-d40c3997a31d-kube-api-access-pntjh\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:38:09.990189 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.990150 2555 generic.go:356] "Generic (PLEG): container finished" podID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerID="56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c" exitCode=0 Jul 06 19:38:09.990362 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.990205 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" Jul 06 19:38:09.990362 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.990239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerDied","Data":"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c"} Jul 06 19:38:09.990362 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.990294 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-f5b8r" event={"ID":"ef5a4c0f-818a-4517-b35a-d40c3997a31d","Type":"ContainerDied","Data":"6fd49f9ddcfb013ed5ce91835e51e0c7294b82712c9741e15f29de7b7214b71c"} Jul 06 19:38:09.990362 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.990319 2555 scope.go:117] "RemoveContainer" containerID="56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c" Jul 06 19:38:09.999828 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:09.999811 2555 scope.go:117] "RemoveContainer" containerID="46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052" Jul 06 19:38:10.007575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.007559 2555 scope.go:117] "RemoveContainer" containerID="56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c" Jul 06 19:38:10.007872 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:38:10.007854 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c\": container with ID starting with 56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c not found: ID does not exist" containerID="56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c" Jul 06 19:38:10.007936 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.007882 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c"} err="failed to get container status \"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c\": rpc error: code = NotFound desc = could not find container \"56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c\": container with ID starting with 56af4a124191c4ce4ca9d9e1a603f83af1132fca341e723e66be6149c0cb9b5c not found: ID does not exist" Jul 06 19:38:10.007936 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.007900 2555 scope.go:117] "RemoveContainer" containerID="46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052" Jul 06 19:38:10.008137 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:38:10.008121 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052\": container with ID starting with 46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052 not found: ID does not exist" containerID="46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052" Jul 06 19:38:10.008177 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.008145 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052"} err="failed to get container status \"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052\": rpc error: code = NotFound desc = could not find container \"46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052\": container with ID starting with 46bd4d9b0b0c2ea8fce10958410c2fa9c5a43c41f9b001807e2cc747c9ab5052 not found: ID does not exist" Jul 06 19:38:10.014868 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.014846 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:38:10.018873 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.018849 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-f5b8r"] Jul 06 19:38:10.669686 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.669645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:10.779193 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.779155 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746f49cdbd-qzxdk"] Jul 06 19:38:10.779735 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.779711 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:38:10.779735 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.779733 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:38:10.779957 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.779871 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:38:10.784277 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.784256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:10.788426 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.788399 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:38:10.788555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.788411 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:38:10.788555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.788465 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tlz9n\"" Jul 06 19:38:10.794030 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.794003 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-qzxdk"] Jul 06 19:38:10.936996 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.936914 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhn87\" (UniqueName: \"kubernetes.io/projected/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-kube-api-access-hhn87\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:10.936996 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.936988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-webhook-cert\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:10.970790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:10.970760 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" path="/var/lib/kubelet/pods/ef5a4c0f-818a-4517-b35a-d40c3997a31d/volumes" Jul 06 19:38:11.038292 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.038259 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhn87\" (UniqueName: \"kubernetes.io/projected/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-kube-api-access-hhn87\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:11.038730 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.038325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-webhook-cert\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:11.040944 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.040925 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-webhook-cert\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:11.047935 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.047896 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhn87\" (UniqueName: \"kubernetes.io/projected/4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4-kube-api-access-hhn87\") pod \"maas-controller-746f49cdbd-qzxdk\" (UID: \"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4\") " pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:11.095966 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.095928 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:11.242349 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:11.242327 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746f49cdbd-qzxdk"] Jul 06 19:38:11.244264 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:38:11.244238 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab7fe55_ec24_4cbe_8aa5_4e658cde87f4.slice/crio-a29f12b57add9c3c0c2d57239ead3ed37e8aa96de8e1c2333e8cd2fb3c01c3ca WatchSource:0}: Error finding container a29f12b57add9c3c0c2d57239ead3ed37e8aa96de8e1c2333e8cd2fb3c01c3ca: Status 404 returned error can't find the container with id a29f12b57add9c3c0c2d57239ead3ed37e8aa96de8e1c2333e8cd2fb3c01c3ca Jul 06 19:38:12.002741 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:12.002703 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" event={"ID":"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4","Type":"ContainerStarted","Data":"b74b45fc83e9a4305590df565aac4e8bbac8d3b3d02d4f06f62ef9cb19ee442d"} Jul 06 19:38:12.002927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:12.002761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" event={"ID":"4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4","Type":"ContainerStarted","Data":"a29f12b57add9c3c0c2d57239ead3ed37e8aa96de8e1c2333e8cd2fb3c01c3ca"} Jul 06 19:38:12.002927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:12.002800 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:14.880029 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:14.879970 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" podStartSLOduration=4.433773356 podStartE2EDuration="4.879954291s" totalImagesPullingTime="446.180935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:14.876445518 +0000 UTC m=+876.382044005" watchObservedRunningTime="2026-07-06 19:38:14.879954291 +0000 UTC m=+876.385552797" Jul 06 19:38:24.009570 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:24.009522 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" podUID="4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:38:25.745324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:25.745290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:25.929155 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:25.929120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:33.012563 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:33.012486 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" Jul 06 19:38:54.191736 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:54.191690 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:54.253310 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:54.253263 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:54.424921 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:38:54.424887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:00.791574 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:00.791533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:00.852696 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:00.852658 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:00.915980 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:00.915943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:01.022496 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:01.022466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:01.145974 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:01.145891 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:07.126866 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:07.126836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:07.185768 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:39:07.185729 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:29.447861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:41:29.447769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:30.859611 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:41:30.859567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.444230 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:42:08.444189 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:20.964281 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:43:20.964247 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:21.829705 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:43:21.829662 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:46.339053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:43:46.339007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:24.588155 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:44:24.588116 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:58.581774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:46:58.581711 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:59.811917 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:46:59.811884 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:24.008938 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:48:24.008898 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" podUID="4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:49:19.751191 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:49:19.751098 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:24.724723 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:49:24.724680 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:06.250936 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:06.250899 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:17.378810 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:17.378775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:25.619297 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:25.619258 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:30.700561 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:30.700473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:33.608558 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:33.608522 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:33.699916 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:33.699877 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:34.018221 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:34.018184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:40.217809 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:40.217768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:40.275269 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:40.275232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:40.449274 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:40.449237 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:40.511053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:40.510962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:41.560393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.560356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:41.643674 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.643641 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:41.644068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.644055 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:50:41.644068 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.644069 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:50:41.644171 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.644135 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5a4c0f-818a-4517-b35a-d40c3997a31d" containerName="manager" Jul 06 19:50:41.645448 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.645417 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:41.647383 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.647367 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.650733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.650714 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b1cbd1f8-serving-cert\"" Jul 06 19:50:41.659395 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.659372 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:41.723685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.723653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.723863 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.723716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.723863 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.723761 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpb48\" (UniqueName: \"kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.746846 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.746802 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:41.824838 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.824745 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.824838 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.824804 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.825215 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.824842 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpb48\" (UniqueName: \"kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.825215 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:50:41.824932 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b1cbd1f8-serving-cert: secret "maas-api-e2e-ait-b1cbd1f8-serving-cert" not found Jul 06 19:50:41.825215 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:50:41.825013 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls podName:6e1acd19-1479-4b4c-8361-4fca6fd5439e nodeName:}" failed. No retries permitted until 2026-07-06 19:50:42.324991958 +0000 UTC m=+1623.830590420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls") pod "maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" (UID: "6e1acd19-1479-4b4c-8361-4fca6fd5439e") : secret "maas-api-e2e-ait-b1cbd1f8-serving-cert" not found Jul 06 19:50:41.825792 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.825764 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.834322 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.834288 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpb48\" (UniqueName: \"kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:41.933929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:41.933896 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:42.232273 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.232235 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:42.329617 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.329567 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:42.332106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.332075 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") pod \"maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:42.336569 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.336500 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:42.560266 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.560186 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:42.703256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.703227 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:42.706032 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:50:42.706002 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1acd19_1479_4b4c_8361_4fca6fd5439e.slice/crio-3e7dd9fdc5786f8f32d0242f5f94f4d20bec6a5342f2d2925677e1ee44073e6c WatchSource:0}: Error finding container 3e7dd9fdc5786f8f32d0242f5f94f4d20bec6a5342f2d2925677e1ee44073e6c: Status 404 returned error can't find the container with id 3e7dd9fdc5786f8f32d0242f5f94f4d20bec6a5342f2d2925677e1ee44073e6c Jul 06 19:50:42.707540 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:42.707522 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:50:43.632183 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:43.632150 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" event={"ID":"6e1acd19-1479-4b4c-8361-4fca6fd5439e","Type":"ContainerStarted","Data":"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717"} Jul 06 19:50:43.632376 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:43.632197 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" event={"ID":"6e1acd19-1479-4b4c-8361-4fca6fd5439e","Type":"ContainerStarted","Data":"3e7dd9fdc5786f8f32d0242f5f94f4d20bec6a5342f2d2925677e1ee44073e6c"} Jul 06 19:50:43.632376 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:43.632282 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:43.655489 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:43.655441 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" podStartSLOduration=2.212551676 podStartE2EDuration="2.655424078s" totalImagesPullingTime="442.872402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:43.652466411 +0000 UTC m=+1625.158064898" watchObservedRunningTime="2026-07-06 19:50:43.655424078 +0000 UTC m=+1625.161022560" Jul 06 19:50:44.732163 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:44.732128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:46.702737 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:46.702705 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:46.882436 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:46.882402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:46.983127 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:46.983040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:47.151585 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.151551 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:47.151810 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.151770 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" podUID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" containerName="maas-api" containerID="cri-o://0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717" gracePeriod=30 Jul 06 19:50:47.204840 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.204809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:47.400118 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.400096 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:47.482772 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.482744 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca\") pod \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " Jul 06 19:50:47.482984 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.482855 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpb48\" (UniqueName: \"kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48\") pod \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " Jul 06 19:50:47.482984 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.482972 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") pod \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\" (UID: \"6e1acd19-1479-4b4c-8361-4fca6fd5439e\") " Jul 06 19:50:47.483178 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.483150 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e1acd19-1479-4b4c-8361-4fca6fd5439e" (UID: "6e1acd19-1479-4b4c-8361-4fca6fd5439e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:50:47.483266 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.483251 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1acd19-1479-4b4c-8361-4fca6fd5439e-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:50:47.485113 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.485091 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48" (OuterVolumeSpecName: "kube-api-access-dpb48") pod "6e1acd19-1479-4b4c-8361-4fca6fd5439e" (UID: "6e1acd19-1479-4b4c-8361-4fca6fd5439e"). InnerVolumeSpecName "kube-api-access-dpb48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:50:47.485244 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.485224 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e1acd19-1479-4b4c-8361-4fca6fd5439e" (UID: "6e1acd19-1479-4b4c-8361-4fca6fd5439e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:50:47.584610 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.584495 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e1acd19-1479-4b4c-8361-4fca6fd5439e-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:50:47.584610 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.584527 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpb48\" (UniqueName: \"kubernetes.io/projected/6e1acd19-1479-4b4c-8361-4fca6fd5439e-kube-api-access-dpb48\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:50:47.655813 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.655777 2555 generic.go:356] "Generic (PLEG): container finished" podID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" containerID="0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717" exitCode=0 Jul 06 19:50:47.655989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.655819 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" Jul 06 19:50:47.655989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.655851 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" event={"ID":"6e1acd19-1479-4b4c-8361-4fca6fd5439e","Type":"ContainerDied","Data":"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717"} Jul 06 19:50:47.655989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.655897 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc" event={"ID":"6e1acd19-1479-4b4c-8361-4fca6fd5439e","Type":"ContainerDied","Data":"3e7dd9fdc5786f8f32d0242f5f94f4d20bec6a5342f2d2925677e1ee44073e6c"} Jul 06 19:50:47.655989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.655921 2555 scope.go:117] "RemoveContainer" containerID="0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717" Jul 06 19:50:47.666855 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.666835 2555 scope.go:117] "RemoveContainer" containerID="0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717" Jul 06 19:50:47.667159 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:50:47.667138 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717\": container with ID starting with 0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717 not found: ID does not exist" containerID="0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717" Jul 06 19:50:47.667233 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.667173 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717"} err="failed to get container status \"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717\": rpc error: code = NotFound desc = could not find container \"0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717\": container with ID starting with 0f725cd0fc95b706e30a9b513b10a5671dddcf314dd48b20fc1dfb030bbf4717 not found: ID does not exist" Jul 06 19:50:47.690321 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.690289 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:47.699538 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:47.699511 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b1cbd1f8-7dfc49b6b6-lsgbc"] Jul 06 19:50:48.439993 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:48.439961 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:48.967695 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:48.967661 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" path="/var/lib/kubelet/pods/6e1acd19-1479-4b4c-8361-4fca6fd5439e/volumes" Jul 06 19:50:53.876543 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:53.876501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:53.979181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:53.979119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:54.202184 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:54.202144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:54.295126 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:54.295076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.349393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.349353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.466222 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.466188 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.545029 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.544995 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:50:55.545896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.545870 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" containerName="maas-api" Jul 06 19:50:55.545896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.545895 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" containerName="maas-api" Jul 06 19:50:55.546133 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.546024 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1acd19-1479-4b4c-8361-4fca6fd5439e" containerName="maas-api" Jul 06 19:50:55.550691 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.550669 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.553606 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.553568 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-174709cd-serving-cert\"" Jul 06 19:50:55.567854 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.567831 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:50:55.641123 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.641027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.658915 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.658888 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.659094 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.658949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.659159 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.659096 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-995nd\" (UniqueName: \"kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.760728 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.760686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-995nd\" (UniqueName: \"kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.760924 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.760745 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.760924 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.760781 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.760924 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:50:55.760860 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-174709cd-serving-cert: secret "maas-api-e2e-ait-174709cd-serving-cert" not found Jul 06 19:50:55.761113 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:50:55.760947 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls podName:1bff00ab-06ce-4af7-8c96-5f1b252adf03 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:56.260925426 +0000 UTC m=+1637.766523889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls") pod "maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" (UID: "1bff00ab-06ce-4af7-8c96-5f1b252adf03") : secret "maas-api-e2e-ait-174709cd-serving-cert" not found Jul 06 19:50:55.761450 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.761430 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:55.773989 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:55.773961 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-995nd\" (UniqueName: \"kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:56.246171 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.246132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:56.266326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.266299 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:56.268831 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.268803 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") pod \"maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:56.349297 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.349266 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:56.452177 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.452128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:56.464027 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.464003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:56.619911 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.617892 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:50:56.701294 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:56.701261 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" event={"ID":"1bff00ab-06ce-4af7-8c96-5f1b252adf03","Type":"ContainerStarted","Data":"709706a7544c1f46ff705bb3dbc80e4a676846deac5e2f6d1bb88cd3105a43af"} Jul 06 19:50:57.589031 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:57.588983 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:57.708564 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:57.708531 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" event={"ID":"1bff00ab-06ce-4af7-8c96-5f1b252adf03","Type":"ContainerStarted","Data":"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030"} Jul 06 19:50:57.708753 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:57.708724 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:50:57.765383 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:50:57.765330 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" podStartSLOduration=2.015721737 podStartE2EDuration="2.76531273s" totalImagesPullingTime="749.590993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:57.763517678 +0000 UTC m=+1639.269116161" watchObservedRunningTime="2026-07-06 19:50:57.76531273 +0000 UTC m=+1639.270911212" Jul 06 19:51:00.462003 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:00.461965 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:00.896080 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:00.896003 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:51:00.896242 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:00.896194 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" podUID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" containerName="maas-api" containerID="cri-o://71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030" gracePeriod=30 Jul 06 19:51:00.915740 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:00.915695 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:01.158224 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.158168 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:51:01.213902 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.213874 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca\") pod \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " Jul 06 19:51:01.214061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.213909 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-995nd\" (UniqueName: \"kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd\") pod \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " Jul 06 19:51:01.214061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.213979 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") pod \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\" (UID: \"1bff00ab-06ce-4af7-8c96-5f1b252adf03\") " Jul 06 19:51:01.214233 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.214209 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1bff00ab-06ce-4af7-8c96-5f1b252adf03" (UID: "1bff00ab-06ce-4af7-8c96-5f1b252adf03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:01.216092 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.216069 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1bff00ab-06ce-4af7-8c96-5f1b252adf03" (UID: "1bff00ab-06ce-4af7-8c96-5f1b252adf03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:01.216185 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.216148 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd" (OuterVolumeSpecName: "kube-api-access-995nd") pod "1bff00ab-06ce-4af7-8c96-5f1b252adf03" (UID: "1bff00ab-06ce-4af7-8c96-5f1b252adf03"). InnerVolumeSpecName "kube-api-access-995nd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:01.314551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.314518 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bff00ab-06ce-4af7-8c96-5f1b252adf03-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:01.314551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.314546 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bff00ab-06ce-4af7-8c96-5f1b252adf03-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:01.314551 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.314557 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-995nd\" (UniqueName: \"kubernetes.io/projected/1bff00ab-06ce-4af7-8c96-5f1b252adf03-kube-api-access-995nd\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:01.728347 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.728307 2555 generic.go:356] "Generic (PLEG): container finished" podID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" containerID="71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030" exitCode=0 Jul 06 19:51:01.728836 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.728352 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" Jul 06 19:51:01.728836 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.728394 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" event={"ID":"1bff00ab-06ce-4af7-8c96-5f1b252adf03","Type":"ContainerDied","Data":"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030"} Jul 06 19:51:01.728836 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.728493 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv" event={"ID":"1bff00ab-06ce-4af7-8c96-5f1b252adf03","Type":"ContainerDied","Data":"709706a7544c1f46ff705bb3dbc80e4a676846deac5e2f6d1bb88cd3105a43af"} Jul 06 19:51:01.728836 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.728526 2555 scope.go:117] "RemoveContainer" containerID="71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030" Jul 06 19:51:01.738209 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.738193 2555 scope.go:117] "RemoveContainer" containerID="71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030" Jul 06 19:51:01.738446 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:51:01.738426 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030\": container with ID starting with 71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030 not found: ID does not exist" containerID="71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030" Jul 06 19:51:01.738489 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.738454 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030"} err="failed to get container status \"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030\": rpc error: code = NotFound desc = could not find container \"71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030\": container with ID starting with 71891999d1d1090bbd13360a9c33eba1752e13b332bab6d3831d329ef800d030 not found: ID does not exist" Jul 06 19:51:01.759156 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.759120 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:51:01.767573 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:01.767548 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-174709cd-6dfd47dc46-pmqdv"] Jul 06 19:51:02.967188 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:02.967155 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" path="/var/lib/kubelet/pods/1bff00ab-06ce-4af7-8c96-5f1b252adf03/volumes" Jul 06 19:51:06.500569 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:06.500523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:07.838613 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:07.838563 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:08.333367 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:08.333323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:15.093575 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:15.093535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:15.163853 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:15.163819 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:15.375354 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:15.375268 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:15.482856 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:15.482814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:16.587748 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.587709 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:16.685385 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.685338 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:16.721161 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.721130 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:16.721685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.721666 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" containerName="maas-api" Jul 06 19:51:16.721685 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.721687 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" containerName="maas-api" Jul 06 19:51:16.721850 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.721778 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bff00ab-06ce-4af7-8c96-5f1b252adf03" containerName="maas-api" Jul 06 19:51:16.725291 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.725269 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.727900 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.727876 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-22b142fd-serving-cert\"" Jul 06 19:51:16.769007 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.768980 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:16.818880 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.818839 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:16.858537 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.858457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spfqv\" (UniqueName: \"kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.858537 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.858510 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.858781 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.858617 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.959903 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.959864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.960129 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.959955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.960129 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.960045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spfqv\" (UniqueName: \"kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.960687 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.960660 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.962654 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.962632 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.985758 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.985728 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spfqv\" (UniqueName: \"kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv\") pod \"maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:16.986053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:16.986027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:17.036457 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.036418 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:17.192879 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.192854 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:17.194582 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:51:17.194550 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d326330_c762_440f_a0b2_d8e5737a5bea.slice/crio-e0c0245bb1c59003e15c9a6c3e6dc12a8b0d6e98d92f99f2cc502550ee5be06b WatchSource:0}: Error finding container e0c0245bb1c59003e15c9a6c3e6dc12a8b0d6e98d92f99f2cc502550ee5be06b: Status 404 returned error can't find the container with id e0c0245bb1c59003e15c9a6c3e6dc12a8b0d6e98d92f99f2cc502550ee5be06b Jul 06 19:51:17.811283 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.811247 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" event={"ID":"6d326330-c762-440f-a0b2-d8e5737a5bea","Type":"ContainerStarted","Data":"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026"} Jul 06 19:51:17.811701 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.811307 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" event={"ID":"6d326330-c762-440f-a0b2-d8e5737a5bea","Type":"ContainerStarted","Data":"e0c0245bb1c59003e15c9a6c3e6dc12a8b0d6e98d92f99f2cc502550ee5be06b"} Jul 06 19:51:17.811701 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.811345 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:17.839313 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.839252 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" podStartSLOduration=1.425502998 podStartE2EDuration="1.839234993s" totalImagesPullingTime="413.731995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:17.837468109 +0000 UTC m=+1659.343066591" watchObservedRunningTime="2026-07-06 19:51:17.839234993 +0000 UTC m=+1659.344833510" Jul 06 19:51:17.855982 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.855917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:17.939198 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:17.939094 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:18.793567 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:18.793534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:20.960112 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:20.960077 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:21.174166 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.174125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:21.282851 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.282735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:21.505555 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.505522 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:21.505783 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.505759 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" podUID="6d326330-c762-440f-a0b2-d8e5737a5bea" containerName="maas-api" containerID="cri-o://46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026" gracePeriod=30 Jul 06 19:51:21.534300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.534219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:21.756272 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.756248 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:21.832932 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.832851 2555 generic.go:356] "Generic (PLEG): container finished" podID="6d326330-c762-440f-a0b2-d8e5737a5bea" containerID="46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026" exitCode=0 Jul 06 19:51:21.832932 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.832889 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" event={"ID":"6d326330-c762-440f-a0b2-d8e5737a5bea","Type":"ContainerDied","Data":"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026"} Jul 06 19:51:21.832932 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.832905 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" Jul 06 19:51:21.832932 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.832929 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj" event={"ID":"6d326330-c762-440f-a0b2-d8e5737a5bea","Type":"ContainerDied","Data":"e0c0245bb1c59003e15c9a6c3e6dc12a8b0d6e98d92f99f2cc502550ee5be06b"} Jul 06 19:51:21.833174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.832951 2555 scope.go:117] "RemoveContainer" containerID="46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026" Jul 06 19:51:21.841566 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.841546 2555 scope.go:117] "RemoveContainer" containerID="46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026" Jul 06 19:51:21.841826 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:51:21.841809 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026\": container with ID starting with 46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026 not found: ID does not exist" containerID="46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026" Jul 06 19:51:21.841878 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.841836 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026"} err="failed to get container status \"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026\": rpc error: code = NotFound desc = could not find container \"46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026\": container with ID starting with 46726d69614c7e0d7f1945b840834989b0ae1b59cf80d8593866fed5bba8d026 not found: ID does not exist" Jul 06 19:51:21.911363 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.911333 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca\") pod \"6d326330-c762-440f-a0b2-d8e5737a5bea\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " Jul 06 19:51:21.911518 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.911377 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spfqv\" (UniqueName: \"kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv\") pod \"6d326330-c762-440f-a0b2-d8e5737a5bea\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " Jul 06 19:51:21.911518 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.911488 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls\") pod \"6d326330-c762-440f-a0b2-d8e5737a5bea\" (UID: \"6d326330-c762-440f-a0b2-d8e5737a5bea\") " Jul 06 19:51:21.911766 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.911736 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d326330-c762-440f-a0b2-d8e5737a5bea" (UID: "6d326330-c762-440f-a0b2-d8e5737a5bea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:21.913574 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.913551 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv" (OuterVolumeSpecName: "kube-api-access-spfqv") pod "6d326330-c762-440f-a0b2-d8e5737a5bea" (UID: "6d326330-c762-440f-a0b2-d8e5737a5bea"). InnerVolumeSpecName "kube-api-access-spfqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:21.913712 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:21.913695 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d326330-c762-440f-a0b2-d8e5737a5bea" (UID: "6d326330-c762-440f-a0b2-d8e5737a5bea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:22.012978 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.012940 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d326330-c762-440f-a0b2-d8e5737a5bea-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:22.012978 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.012971 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d326330-c762-440f-a0b2-d8e5737a5bea-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:22.012978 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.012981 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spfqv\" (UniqueName: \"kubernetes.io/projected/6d326330-c762-440f-a0b2-d8e5737a5bea-kube-api-access-spfqv\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:51:22.163303 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.163258 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:22.165332 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.165307 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-22b142fd-9b7748bc5-xzwvj"] Jul 06 19:51:22.879802 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.879750 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:22.973329 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:22.973288 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d326330-c762-440f-a0b2-d8e5737a5bea" path="/var/lib/kubelet/pods/6d326330-c762-440f-a0b2-d8e5737a5bea/volumes" Jul 06 19:51:27.906496 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:27.906458 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:28.006758 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:28.006724 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:28.412895 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:28.412853 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:28.508711 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:28.508672 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:28.612297 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:28.612265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:30.048790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:30.048755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:37.128437 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:37.128394 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:37.192920 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:37.192882 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:37.321895 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:37.321858 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:37.406211 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:37.406096 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:37.964446 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:51:37.964402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:03.868839 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:03.868753 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:09.687136 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:09.687085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:09.784300 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:09.784265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:10.671568 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:10.671523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:10.769338 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:10.769305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:10.865972 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:10.865936 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:26.785820 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:26.785776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:33.320862 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:33.320823 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:33.422229 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:33.422201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:33.518880 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:33.518846 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:44.017170 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:44.017129 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:44.829645 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:44.829558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:51.215005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:51.214967 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:51.284477 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:51.284444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:51.353666 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:51.353618 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:51.451094 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:51.451061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:51.551380 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:51.551293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:52.271518 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:52.271483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:52.371579 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:52.371552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:52.474037 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:52.473998 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:52:54.175648 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:52:54.175580 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:00.457801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:00.457763 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:00.553180 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:00.553147 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:01.123943 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:01.123895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:07.386097 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:07.386057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:07.489650 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:07.489615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:07.658469 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:07.658387 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:08.026106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:08.026053 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:08.574461 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:08.574426 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:11.218322 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:11.218280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:14.590025 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.589995 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:14.590586 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.590564 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d326330-c762-440f-a0b2-d8e5737a5bea" containerName="maas-api" Jul 06 19:53:14.590586 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.590587 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d326330-c762-440f-a0b2-d8e5737a5bea" containerName="maas-api" Jul 06 19:53:14.590764 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.590734 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d326330-c762-440f-a0b2-d8e5737a5bea" containerName="maas-api" Jul 06 19:53:14.593197 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.593166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:14.594151 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.594131 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.597005 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.596986 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-925fb4a7-serving-cert\"" Jul 06 19:53:14.675829 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.675798 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp7pc\" (UniqueName: \"kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.675988 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.675930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.675988 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.675969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.776988 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.776950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp7pc\" (UniqueName: \"kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.777160 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.777076 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.777160 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.777112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:14.777286 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:53:14.777226 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-925fb4a7-serving-cert: secret "maas-api-e2e-mt-925fb4a7-serving-cert" not found Jul 06 19:53:14.777341 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:53:14.777309 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls podName:5069d936-02ea-4635-a814-e7d58dba1569 nodeName:}" failed. No retries permitted until 2026-07-06 19:53:15.277284839 +0000 UTC m=+1776.782883314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls") pod "maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" (UID: "5069d936-02ea-4635-a814-e7d58dba1569") : secret "maas-api-e2e-mt-925fb4a7-serving-cert" not found Jul 06 19:53:14.777719 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:14.777698 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:15.035454 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.035417 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:15.043125 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.043090 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp7pc\" (UniqueName: \"kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:15.282370 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.282336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:15.284903 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.284882 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") pod \"maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:15.506013 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.505971 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:15.695264 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:15.695223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:16.932801 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:16.932770 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:16.936453 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:53:16.936428 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5069d936_02ea_4635_a814_e7d58dba1569.slice/crio-892fb7d47f15fa81f67ba195bb3dfb8f0484e5e619612e307bd663b0fab1ba4e WatchSource:0}: Error finding container 892fb7d47f15fa81f67ba195bb3dfb8f0484e5e619612e307bd663b0fab1ba4e: Status 404 returned error can't find the container with id 892fb7d47f15fa81f67ba195bb3dfb8f0484e5e619612e307bd663b0fab1ba4e Jul 06 19:53:17.389312 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:17.389277 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" event={"ID":"5069d936-02ea-4635-a814-e7d58dba1569","Type":"ContainerStarted","Data":"892fb7d47f15fa81f67ba195bb3dfb8f0484e5e619612e307bd663b0fab1ba4e"} Jul 06 19:53:18.395285 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:18.395207 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" event={"ID":"5069d936-02ea-4635-a814-e7d58dba1569","Type":"ContainerStarted","Data":"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25"} Jul 06 19:53:18.395649 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:18.395287 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:19.332542 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:19.332492 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:22.290524 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:22.290470 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" podStartSLOduration=9.800109359 podStartE2EDuration="10.290453422s" totalImagesPullingTime="490.344063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:22.288955248 +0000 UTC m=+1783.794553730" watchObservedRunningTime="2026-07-06 19:53:22.290453422 +0000 UTC m=+1783.796051904" Jul 06 19:53:24.009348 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:24.009304 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" podUID="4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:53:24.404816 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:24.404741 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:29.443093 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:29.443010 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:29.624236 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:29.624192 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:29.831961 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:29.831923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:29.931826 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:29.931788 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:30.493110 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:30.493073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:30.594012 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:30.593972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:30.980285 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:30.980250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:37.261158 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:37.261115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:37.371839 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:37.371809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:37.707355 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:37.707324 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:38.022745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.022672 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:38.022902 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.022875 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" podUID="5069d936-02ea-4635-a814-e7d58dba1569" containerName="maas-api" containerID="cri-o://a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25" gracePeriod=30 Jul 06 19:53:38.048840 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.048813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:38.146181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.146152 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:38.275654 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.275578 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:38.401213 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.401182 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp7pc\" (UniqueName: \"kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc\") pod \"5069d936-02ea-4635-a814-e7d58dba1569\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " Jul 06 19:53:38.401387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.401220 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") pod \"5069d936-02ea-4635-a814-e7d58dba1569\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " Jul 06 19:53:38.401387 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.401262 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca\") pod \"5069d936-02ea-4635-a814-e7d58dba1569\" (UID: \"5069d936-02ea-4635-a814-e7d58dba1569\") " Jul 06 19:53:38.401657 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.401631 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5069d936-02ea-4635-a814-e7d58dba1569" (UID: "5069d936-02ea-4635-a814-e7d58dba1569"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:53:38.403441 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.403419 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5069d936-02ea-4635-a814-e7d58dba1569" (UID: "5069d936-02ea-4635-a814-e7d58dba1569"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:53:38.403671 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.403652 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc" (OuterVolumeSpecName: "kube-api-access-tp7pc") pod "5069d936-02ea-4635-a814-e7d58dba1569" (UID: "5069d936-02ea-4635-a814-e7d58dba1569"). InnerVolumeSpecName "kube-api-access-tp7pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:53:38.494913 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.494881 2555 generic.go:356] "Generic (PLEG): container finished" podID="5069d936-02ea-4635-a814-e7d58dba1569" containerID="a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25" exitCode=0 Jul 06 19:53:38.495012 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.494918 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" event={"ID":"5069d936-02ea-4635-a814-e7d58dba1569","Type":"ContainerDied","Data":"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25"} Jul 06 19:53:38.495012 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.494928 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" Jul 06 19:53:38.495012 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.495007 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p" event={"ID":"5069d936-02ea-4635-a814-e7d58dba1569","Type":"ContainerDied","Data":"892fb7d47f15fa81f67ba195bb3dfb8f0484e5e619612e307bd663b0fab1ba4e"} Jul 06 19:53:38.495188 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.495039 2555 scope.go:117] "RemoveContainer" containerID="a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25" Jul 06 19:53:38.502667 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.502654 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp7pc\" (UniqueName: \"kubernetes.io/projected/5069d936-02ea-4635-a814-e7d58dba1569-kube-api-access-tp7pc\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:53:38.502732 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.502671 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5069d936-02ea-4635-a814-e7d58dba1569-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:53:38.502732 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.502682 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5069d936-02ea-4635-a814-e7d58dba1569-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:53:38.504244 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.504224 2555 scope.go:117] "RemoveContainer" containerID="a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25" Jul 06 19:53:38.504475 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:53:38.504456 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25\": container with ID starting with a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25 not found: ID does not exist" containerID="a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25" Jul 06 19:53:38.504518 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.504481 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25"} err="failed to get container status \"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25\": rpc error: code = NotFound desc = could not find container \"a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25\": container with ID starting with a1d7e053525d95fbe09ae1a3c73de563514c40126d06a769b52f897eebf89a25 not found: ID does not exist" Jul 06 19:53:38.518895 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.518873 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:38.520604 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.520574 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-925fb4a7-55fffc97b8-msf2p"] Jul 06 19:53:38.970805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:38.970774 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5069d936-02ea-4635-a814-e7d58dba1569" path="/var/lib/kubelet/pods/5069d936-02ea-4635-a814-e7d58dba1569/volumes" Jul 06 19:53:40.858910 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:40.858872 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:40.960079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:40.960047 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:44.655001 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:44.654965 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:44.754056 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:44.754014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:44.924131 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:44.924044 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:45.132776 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:45.132741 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:45.270934 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:45.270888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:45.359639 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:45.359577 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:45.475348 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:45.475313 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:51.418079 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:51.418032 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:51.475978 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:51.475944 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:51.555096 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:51.555055 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:51.655069 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:51.655040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:51.788015 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:51.787974 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:58.105132 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:58.105087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:58.165190 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:53:58.165163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:04.905412 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:04.905331 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:05.272061 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:05.272013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:06.110285 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:06.110249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:08.534053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.533995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:08.613187 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.613157 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:08.613647 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.613630 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5069d936-02ea-4635-a814-e7d58dba1569" containerName="maas-api" Jul 06 19:54:08.613751 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.613650 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5069d936-02ea-4635-a814-e7d58dba1569" containerName="maas-api" Jul 06 19:54:08.613821 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.613778 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5069d936-02ea-4635-a814-e7d58dba1569" containerName="maas-api" Jul 06 19:54:08.617184 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.617162 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.621336 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.621311 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-08bc2a57-serving-cert\"" Jul 06 19:54:08.630230 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.630208 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:08.649249 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.649207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:08.684557 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.684529 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gwcm\" (UniqueName: \"kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.684721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.684570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.684721 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.684680 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.733056 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.733030 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:08.785677 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.785612 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.785808 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.785728 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gwcm\" (UniqueName: \"kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.785808 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:54:08.785750 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-08bc2a57-serving-cert: secret "maas-api-e2e-mt-08bc2a57-serving-cert" not found Jul 06 19:54:08.785808 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.785777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.786034 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:54:08.785816 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls podName:5cb43c0a-0506-4c33-8d40-3b5ac5ac828a nodeName:}" failed. No retries permitted until 2026-07-06 19:54:09.285796709 +0000 UTC m=+1830.791395190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls") pod "maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" (UID: "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a") : secret "maas-api-e2e-mt-08bc2a57-serving-cert" not found Jul 06 19:54:08.786492 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.786471 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.802547 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.802526 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gwcm\" (UniqueName: \"kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:08.978463 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:08.978433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:09.259850 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.259813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:09.290668 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.290616 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:09.293216 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.293181 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") pod \"maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:09.360357 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.360316 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:09.532805 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.532728 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:09.673585 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:09.673560 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:09.675106 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:54:09.675081 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb43c0a_0506_4c33_8d40_3b5ac5ac828a.slice/crio-3f542dd44591c12f6e476fcd8ea67af316974172f364c82d9e123f3741ae0cb9 WatchSource:0}: Error finding container 3f542dd44591c12f6e476fcd8ea67af316974172f364c82d9e123f3741ae0cb9: Status 404 returned error can't find the container with id 3f542dd44591c12f6e476fcd8ea67af316974172f364c82d9e123f3741ae0cb9 Jul 06 19:54:10.654080 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:10.654044 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" event={"ID":"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a","Type":"ContainerStarted","Data":"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48"} Jul 06 19:54:10.654269 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:10.654102 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" event={"ID":"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a","Type":"ContainerStarted","Data":"3f542dd44591c12f6e476fcd8ea67af316974172f364c82d9e123f3741ae0cb9"} Jul 06 19:54:10.654269 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:10.654135 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:10.674087 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:10.674037 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" podStartSLOduration=2.251272318 podStartE2EDuration="2.674022214s" totalImagesPullingTime="422.749896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:10.67214371 +0000 UTC m=+1832.177742188" watchObservedRunningTime="2026-07-06 19:54:10.674022214 +0000 UTC m=+1832.179620695" Jul 06 19:54:13.064227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:13.064189 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:16.664699 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:16.664664 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:18.511353 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:18.511238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:50.616211 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:50.616162 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:51.032724 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.032688 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:51.032941 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.032903 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" podUID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" containerName="maas-api" containerID="cri-o://cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48" gracePeriod=30 Jul 06 19:54:51.084928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.084884 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:51.286143 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.286082 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:51.388883 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.388845 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") pod \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " Jul 06 19:54:51.389082 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.388925 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gwcm\" (UniqueName: \"kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm\") pod \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " Jul 06 19:54:51.389082 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.388963 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca\") pod \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\" (UID: \"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a\") " Jul 06 19:54:51.389403 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.389372 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" (UID: "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:54:51.391114 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.391090 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" (UID: "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:54:51.391227 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.391195 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm" (OuterVolumeSpecName: "kube-api-access-4gwcm") pod "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" (UID: "5cb43c0a-0506-4c33-8d40-3b5ac5ac828a"). InnerVolumeSpecName "kube-api-access-4gwcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:51.490606 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.490570 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gwcm\" (UniqueName: \"kubernetes.io/projected/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-kube-api-access-4gwcm\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:54:51.490774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.490620 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:54:51.490774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.490630 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:54:51.854738 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.854701 2555 generic.go:356] "Generic (PLEG): container finished" podID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" containerID="cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48" exitCode=0 Jul 06 19:54:51.854738 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.854742 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" Jul 06 19:54:51.855174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.854788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" event={"ID":"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a","Type":"ContainerDied","Data":"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48"} Jul 06 19:54:51.855174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.854838 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s" event={"ID":"5cb43c0a-0506-4c33-8d40-3b5ac5ac828a","Type":"ContainerDied","Data":"3f542dd44591c12f6e476fcd8ea67af316974172f364c82d9e123f3741ae0cb9"} Jul 06 19:54:51.855174 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.854862 2555 scope.go:117] "RemoveContainer" containerID="cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48" Jul 06 19:54:51.864143 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.864127 2555 scope.go:117] "RemoveContainer" containerID="cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48" Jul 06 19:54:51.864417 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:54:51.864398 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48\": container with ID starting with cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48 not found: ID does not exist" containerID="cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48" Jul 06 19:54:51.864487 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.864425 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48"} err="failed to get container status \"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48\": rpc error: code = NotFound desc = could not find container \"cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48\": container with ID starting with cce60d764aa6c52339ea4bab8c6147c81d4bbf080f3db3776919c7608279ab48 not found: ID does not exist" Jul 06 19:54:51.890608 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.890564 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:51.893492 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:51.893473 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-08bc2a57-799d9544b8-2jr2s"] Jul 06 19:54:52.967690 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:52.967658 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" path="/var/lib/kubelet/pods/5cb43c0a-0506-4c33-8d40-3b5ac5ac828a/volumes" Jul 06 19:54:56.725225 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:54:56.725185 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:01.839681 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:01.839551 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:39.666160 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:39.666120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:39.803123 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:39.803085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:46.220028 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:46.219984 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:46.318085 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:46.318040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:46.549444 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:46.549351 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:46.801640 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:46.801522 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:47.870453 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:47.870411 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:47.969160 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:47.969119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:48.069256 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:48.069223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:48.761257 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:48.761225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:48.862968 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:48.862932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:48.968035 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:48.968001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:50.358669 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:50.358632 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:56.829021 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:56.828990 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:56.928494 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:56.928458 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:57.544045 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:55:57.544015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:03.903865 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:03.903830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:03.967193 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:03.967159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:04.075412 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:04.075375 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:04.166252 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:04.166169 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:14.615792 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:14.615745 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:14.950603 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:14.950557 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:35.929013 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:35.928925 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:42.098119 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:42.098084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:42.158484 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:42.158436 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:42.464026 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:42.463988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:42.798081 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:42.797980 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:43.042739 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:43.042703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:46.311441 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.311408 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:46.399519 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.399479 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:56:46.400119 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.400103 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" containerName="maas-api" Jul 06 19:56:46.400173 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.400124 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" containerName="maas-api" Jul 06 19:56:46.400248 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.400236 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb43c0a-0506-4c33-8d40-3b5ac5ac828a" containerName="maas-api" Jul 06 19:56:46.403848 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.403827 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.406050 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.406026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:46.406923 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.406900 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-1e033f-serving-cert\"" Jul 06 19:56:46.416001 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.415981 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:56:46.485366 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.485331 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:46.552770 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.552735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjt4t\" (UniqueName: \"kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.552936 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.552779 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.552936 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.552878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.584986 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.584910 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:46.654270 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.654235 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjt4t\" (UniqueName: \"kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.654447 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.654279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.654447 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.654321 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.655021 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.654991 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.656956 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.656926 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.667517 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.667493 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjt4t\" (UniqueName: \"kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t\") pod \"maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.716309 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.716271 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:46.853251 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.853228 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:56:46.855345 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:56:46.855317 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fb9fe2_2b7b_406e_8e9b_b7a747f46d6a.slice/crio-efd5b0b8ac6cb36299c4d6d71538b4c355e7d13adb436a26a5c006ebc422454e WatchSource:0}: Error finding container efd5b0b8ac6cb36299c4d6d71538b4c355e7d13adb436a26a5c006ebc422454e: Status 404 returned error can't find the container with id efd5b0b8ac6cb36299c4d6d71538b4c355e7d13adb436a26a5c006ebc422454e Jul 06 19:56:46.856733 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:46.856714 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:56:47.027201 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:47.027126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:47.157369 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:47.157270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:47.415788 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:47.415762 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" event={"ID":"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a","Type":"ContainerStarted","Data":"efd5b0b8ac6cb36299c4d6d71538b4c355e7d13adb436a26a5c006ebc422454e"} Jul 06 19:56:48.422521 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:48.422488 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" event={"ID":"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a","Type":"ContainerStarted","Data":"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c"} Jul 06 19:56:48.422934 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:48.422684 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:48.444154 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:48.444116 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" podStartSLOduration=1.895685388 podStartE2EDuration="2.444103919s" totalImagesPullingTime="548.418531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:48.442804029 +0000 UTC m=+1989.948402514" watchObservedRunningTime="2026-07-06 19:56:48.444103919 +0000 UTC m=+1989.949702401" Jul 06 19:56:50.714693 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:50.714651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:51.052244 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:51.052164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:51.375224 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:51.375115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:52.031399 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:52.031365 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.190697 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.190663 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.252465 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.252432 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:56:54.256402 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.256380 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.259010 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.258988 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-64edc6-serving-cert\"" Jul 06 19:56:54.272106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.271523 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:56:54.280048 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.280019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.326208 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.326179 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.326361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.326228 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.326361 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.326281 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2nk\" (UniqueName: \"kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.347232 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.347178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.427213 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.427176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2nk\" (UniqueName: \"kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.427390 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.427279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.427390 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.427320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.427528 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:56:54.427428 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-64edc6-serving-cert: secret "maas-api-e2e-inf-b-64edc6-serving-cert" not found Jul 06 19:56:54.427528 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:56:54.427499 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls podName:a7147179-8adf-4ddd-9207-ac397e83cde3 nodeName:}" failed. No retries permitted until 2026-07-06 19:56:54.927477531 +0000 UTC m=+1996.433075993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls") pod "maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" (UID: "a7147179-8adf-4ddd-9207-ac397e83cde3") : secret "maas-api-e2e-inf-b-64edc6-serving-cert" not found Jul 06 19:56:54.428074 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.428045 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.432454 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.432436 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:56:54.439299 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.439278 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2nk\" (UniqueName: \"kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.646642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.646603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.788383 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.788344 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.890863 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.890827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:54.931923 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.931851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:54.934655 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:54.934628 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") pod \"maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:55.169374 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:55.169324 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:55.312571 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:55.312543 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:56:55.315050 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:56:55.315021 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7147179_8adf_4ddd_9207_ac397e83cde3.slice/crio-b2519847dfcbff82a86c2ae0b06193f61ee7573c54ac0135e0f5aeaca8034a76 WatchSource:0}: Error finding container b2519847dfcbff82a86c2ae0b06193f61ee7573c54ac0135e0f5aeaca8034a76: Status 404 returned error can't find the container with id b2519847dfcbff82a86c2ae0b06193f61ee7573c54ac0135e0f5aeaca8034a76 Jul 06 19:56:55.462454 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:55.462419 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" event={"ID":"a7147179-8adf-4ddd-9207-ac397e83cde3","Type":"ContainerStarted","Data":"b2519847dfcbff82a86c2ae0b06193f61ee7573c54ac0135e0f5aeaca8034a76"} Jul 06 19:56:56.468756 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:56.468718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" event={"ID":"a7147179-8adf-4ddd-9207-ac397e83cde3","Type":"ContainerStarted","Data":"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f"} Jul 06 19:56:56.469180 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:56.468803 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:56:56.497917 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:56.497841 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" podStartSLOduration=2.187289026 podStartE2EDuration="2.497826732s" totalImagesPullingTime="310.537706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:56.496410864 +0000 UTC m=+1998.002009347" watchObservedRunningTime="2026-07-06 19:56:56.497826732 +0000 UTC m=+1998.003425213" Jul 06 19:56:58.700326 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:56:58.700282 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:02.479228 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:02.479201 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:57:07.939100 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:07.939061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:09.139612 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:09.139571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:16.719374 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:16.719320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:16.825478 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:16.825446 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:24.677358 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:57:24.677319 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:24.009150 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:24.009103 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-746f49cdbd-qzxdk" podUID="4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:58:26.026821 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:26.026781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:29.926466 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:29.926426 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:30.031896 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:30.031858 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:38.393150 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:38.393111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:38.463502 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:38.463460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:41.819352 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:41.819308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:41.911755 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:41.911708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:55.902007 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:55.901969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:57.130393 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:57.130358 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:57.313861 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:57.313818 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:57.697929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:57.697890 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:57.855841 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:57.855808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:57.953852 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:57.953744 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:58.134530 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.134499 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:58:58.134935 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.134712 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" podUID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" containerName="maas-api" containerID="cri-o://ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c" gracePeriod=30 Jul 06 19:58:58.159746 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.159701 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:58.387927 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.387902 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:58:58.478921 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.478889 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca\") pod \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " Jul 06 19:58:58.479106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.478977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls\") pod \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " Jul 06 19:58:58.479106 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.479002 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjt4t\" (UniqueName: \"kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t\") pod \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\" (UID: \"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a\") " Jul 06 19:58:58.479230 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.479209 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" (UID: "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:58:58.481324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.481295 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t" (OuterVolumeSpecName: "kube-api-access-zjt4t") pod "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" (UID: "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a"). InnerVolumeSpecName "kube-api-access-zjt4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:58:58.481324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.481311 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" (UID: "71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:58:58.579788 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.579699 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:58:58.579788 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.579730 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:58:58.579788 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:58.579740 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjt4t\" (UniqueName: \"kubernetes.io/projected/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a-kube-api-access-zjt4t\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:58:59.072140 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.072104 2555 generic.go:356] "Generic (PLEG): container finished" podID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" containerID="ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c" exitCode=0 Jul 06 19:58:59.072316 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.072148 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" Jul 06 19:58:59.072316 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.072200 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" event={"ID":"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a","Type":"ContainerDied","Data":"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c"} Jul 06 19:58:59.072316 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.072252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw" event={"ID":"71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a","Type":"ContainerDied","Data":"efd5b0b8ac6cb36299c4d6d71538b4c355e7d13adb436a26a5c006ebc422454e"} Jul 06 19:58:59.072316 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.072273 2555 scope.go:117] "RemoveContainer" containerID="ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c" Jul 06 19:58:59.081438 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.081422 2555 scope.go:117] "RemoveContainer" containerID="ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c" Jul 06 19:58:59.081773 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:58:59.081752 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c\": container with ID starting with ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c not found: ID does not exist" containerID="ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c" Jul 06 19:58:59.081833 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.081781 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c"} err="failed to get container status \"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c\": rpc error: code = NotFound desc = could not find container \"ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c\": container with ID starting with ab50c3ca758ce79a541267fa2bc533b5cba7c89a99aa14b696f7aa4f3fb0da8c not found: ID does not exist" Jul 06 19:58:59.095273 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.095250 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:58:59.099906 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:58:59.099885 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1e033f-56bc85d644-k9tjw"] Jul 06 19:59:00.973260 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:00.973229 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" path="/var/lib/kubelet/pods/71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a/volumes" Jul 06 19:59:02.760053 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:02.760013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:03.245438 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:03.245405 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:40.022560 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:40.022516 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:40.304645 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:40.304526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:40.522703 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:40.522669 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:40.684928 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:40.684838 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:41.090997 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.090962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:41.191166 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.191126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:41.289788 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.289750 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:41.407364 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.407279 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:59:41.407528 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.407482 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" podUID="a7147179-8adf-4ddd-9207-ac397e83cde3" containerName="maas-api" containerID="cri-o://334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f" gracePeriod=30 Jul 06 19:59:41.439135 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.439110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:41.654159 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.654135 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:59:41.673392 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.673326 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca\") pod \"a7147179-8adf-4ddd-9207-ac397e83cde3\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " Jul 06 19:59:41.673509 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.673415 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") pod \"a7147179-8adf-4ddd-9207-ac397e83cde3\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " Jul 06 19:59:41.673509 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.673459 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj2nk\" (UniqueName: \"kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk\") pod \"a7147179-8adf-4ddd-9207-ac397e83cde3\" (UID: \"a7147179-8adf-4ddd-9207-ac397e83cde3\") " Jul 06 19:59:41.673653 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.673632 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7147179-8adf-4ddd-9207-ac397e83cde3" (UID: "a7147179-8adf-4ddd-9207-ac397e83cde3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:59:41.673773 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.673753 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7147179-8adf-4ddd-9207-ac397e83cde3-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:59:41.675573 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.675549 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7147179-8adf-4ddd-9207-ac397e83cde3" (UID: "a7147179-8adf-4ddd-9207-ac397e83cde3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:59:41.675679 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.675583 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk" (OuterVolumeSpecName: "kube-api-access-dj2nk") pod "a7147179-8adf-4ddd-9207-ac397e83cde3" (UID: "a7147179-8adf-4ddd-9207-ac397e83cde3"). InnerVolumeSpecName "kube-api-access-dj2nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:59:41.774386 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.774357 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7147179-8adf-4ddd-9207-ac397e83cde3-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:59:41.774386 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:41.774383 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj2nk\" (UniqueName: \"kubernetes.io/projected/a7147179-8adf-4ddd-9207-ac397e83cde3-kube-api-access-dj2nk\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 19:59:42.294790 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.294758 2555 generic.go:356] "Generic (PLEG): container finished" podID="a7147179-8adf-4ddd-9207-ac397e83cde3" containerID="334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f" exitCode=0 Jul 06 19:59:42.295210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.294799 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" event={"ID":"a7147179-8adf-4ddd-9207-ac397e83cde3","Type":"ContainerDied","Data":"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f"} Jul 06 19:59:42.295210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.294822 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" Jul 06 19:59:42.295210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.294860 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8" event={"ID":"a7147179-8adf-4ddd-9207-ac397e83cde3","Type":"ContainerDied","Data":"b2519847dfcbff82a86c2ae0b06193f61ee7573c54ac0135e0f5aeaca8034a76"} Jul 06 19:59:42.295210 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.294880 2555 scope.go:117] "RemoveContainer" containerID="334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f" Jul 06 19:59:42.304580 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.304561 2555 scope.go:117] "RemoveContainer" containerID="334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f" Jul 06 19:59:42.304855 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:59:42.304833 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f\": container with ID starting with 334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f not found: ID does not exist" containerID="334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f" Jul 06 19:59:42.304926 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.304868 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f"} err="failed to get container status \"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f\": rpc error: code = NotFound desc = could not find container \"334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f\": container with ID starting with 334ce7adb4fd0e26e017687303b0329409f1af973e399d7da53488499fbe386f not found: ID does not exist" Jul 06 19:59:42.318468 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.318434 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:59:42.327501 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.327475 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-64edc6-54fc6d9c77-b2tr8"] Jul 06 19:59:42.968138 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:42.968107 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7147179-8adf-4ddd-9207-ac397e83cde3" path="/var/lib/kubelet/pods/a7147179-8adf-4ddd-9207-ac397e83cde3/volumes" Jul 06 19:59:47.816322 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:47.816292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:47.874233 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:47.874203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:48.090036 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:48.089951 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:48.317776 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:48.317742 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:48.678642 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:48.678584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:48.991929 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:48.991883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:49.237464 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:49.237427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:51.999426 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:51.999379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:52.083843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.083803 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 19:59:52.084280 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084262 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7147179-8adf-4ddd-9207-ac397e83cde3" containerName="maas-api" Jul 06 19:59:52.084375 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084283 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7147179-8adf-4ddd-9207-ac397e83cde3" containerName="maas-api" Jul 06 19:59:52.084375 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084311 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" containerName="maas-api" Jul 06 19:59:52.084375 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084321 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" containerName="maas-api" Jul 06 19:59:52.084529 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084446 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7147179-8adf-4ddd-9207-ac397e83cde3" containerName="maas-api" Jul 06 19:59:52.084529 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.084459 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="71fb9fe2-2b7b-406e-8e9b-b7a747f46d6a" containerName="maas-api" Jul 06 19:59:52.087634 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.087578 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.091031 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.091012 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-c0c2c4-serving-cert\"" Jul 06 19:59:52.093979 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.093952 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:52.106670 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.106648 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 19:59:52.173635 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.173577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8hvx\" (UniqueName: \"kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.173843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.173743 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.173843 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.173795 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.174365 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.174339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:52.274880 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.274796 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.274880 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.274841 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.274880 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.274869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8hvx\" (UniqueName: \"kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.275161 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:59:52.274960 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-c0c2c4-serving-cert: secret "maas-api-e2e-api-a-c0c2c4-serving-cert" not found Jul 06 19:59:52.275161 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:59:52.275050 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls podName:468b230d-56c4-467c-91ab-d8e8db633fcb nodeName:}" failed. No retries permitted until 2026-07-06 19:59:52.775028835 +0000 UTC m=+2174.280627304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls") pod "maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" (UID: "468b230d-56c4-467c-91ab-d8e8db633fcb") : secret "maas-api-e2e-api-a-c0c2c4-serving-cert" not found Jul 06 19:59:52.275683 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.275661 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.287745 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.287724 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8hvx\" (UniqueName: \"kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.710309 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.710266 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:52.780317 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.780290 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:52.780499 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:59:52.780445 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-c0c2c4-serving-cert: secret "maas-api-e2e-api-a-c0c2c4-serving-cert" not found Jul 06 19:59:52.780546 ip-10-0-133-207 kubenswrapper[2555]: E0706 19:59:52.780516 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls podName:468b230d-56c4-467c-91ab-d8e8db633fcb nodeName:}" failed. No retries permitted until 2026-07-06 19:59:53.780499539 +0000 UTC m=+2175.286097999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls") pod "maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" (UID: "468b230d-56c4-467c-91ab-d8e8db633fcb") : secret "maas-api-e2e-api-a-c0c2c4-serving-cert" not found Jul 06 19:59:52.815402 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.815349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:52.918655 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:52.918615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:53.791282 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:53.791248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:53.793799 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:53.793778 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:53.898358 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:53.898322 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:54.036186 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:54.036163 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 19:59:54.038358 ip-10-0-133-207 kubenswrapper[2555]: W0706 19:59:54.038332 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468b230d_56c4_467c_91ab_d8e8db633fcb.slice/crio-cb5a54e1e651b0c8a5422494e9c09c79ca3efed1dc9b231cb96052c869d7a764 WatchSource:0}: Error finding container cb5a54e1e651b0c8a5422494e9c09c79ca3efed1dc9b231cb96052c869d7a764: Status 404 returned error can't find the container with id cb5a54e1e651b0c8a5422494e9c09c79ca3efed1dc9b231cb96052c869d7a764 Jul 06 19:59:54.353822 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:54.353733 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" event={"ID":"468b230d-56c4-467c-91ab-d8e8db633fcb","Type":"ContainerStarted","Data":"cb5a54e1e651b0c8a5422494e9c09c79ca3efed1dc9b231cb96052c869d7a764"} Jul 06 19:59:55.359935 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:55.359894 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" event={"ID":"468b230d-56c4-467c-91ab-d8e8db633fcb","Type":"ContainerStarted","Data":"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87"} Jul 06 19:59:55.360324 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:55.360047 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 19:59:55.382315 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:55.382254 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" podStartSLOduration=2.8020898 podStartE2EDuration="3.3822387s" totalImagesPullingTime="580.1489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:55.379519719 +0000 UTC m=+2176.885118202" watchObservedRunningTime="2026-07-06 19:59:55.3822387 +0000 UTC m=+2176.887837182" Jul 06 19:59:56.499181 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:56.499145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:56.957774 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:56.957739 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:57.285909 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:57.285817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:57.580335 ip-10-0-133-207 kubenswrapper[2555]: I0706 19:59:57.580247 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.044572 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.044535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.119053 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.119020 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:00.123007 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.122980 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.124216 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.124184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.125879 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.125858 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-bc0ca2-serving-cert\"" Jul 06 20:00:00.137169 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.137137 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:00.216302 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.216264 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.255372 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.255336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.255573 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.255437 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q2x6\" (UniqueName: \"kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.255573 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.255466 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.356327 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.356233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q2x6\" (UniqueName: \"kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.356327 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.356287 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.356327 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.356326 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.356937 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.356914 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.358890 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.358868 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.367479 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.367454 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q2x6\" (UniqueName: \"kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6\") pod \"maas-api-e2e-api-b-bc0ca2-6455746755-kd55v\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.373420 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.373395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.436860 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.436823 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:00.575411 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.575384 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:00.578079 ip-10-0-133-207 kubenswrapper[2555]: W0706 20:00:00.578040 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404c0867_fc6e_4105_9b3f_07fac496a3e6.slice/crio-5cac56b594de9aeffb027cbc82320b9d7b952536040425ae9d9720e95afb0831 WatchSource:0}: Error finding container 5cac56b594de9aeffb027cbc82320b9d7b952536040425ae9d9720e95afb0831: Status 404 returned error can't find the container with id 5cac56b594de9aeffb027cbc82320b9d7b952536040425ae9d9720e95afb0831 Jul 06 20:00:00.777028 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.776997 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:00.881544 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:00.881514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:01.370356 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:01.370330 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 20:00:01.392504 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:01.392471 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" event={"ID":"404c0867-fc6e-4105-9b3f-07fac496a3e6","Type":"ContainerStarted","Data":"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2"} Jul 06 20:00:01.392701 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:01.392520 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" event={"ID":"404c0867-fc6e-4105-9b3f-07fac496a3e6","Type":"ContainerStarted","Data":"5cac56b594de9aeffb027cbc82320b9d7b952536040425ae9d9720e95afb0831"} Jul 06 20:00:01.392701 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:01.392623 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:01.418454 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:01.418387 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" podStartSLOduration=1.034126866 podStartE2EDuration="1.418363414s" totalImagesPullingTime="384.236548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:01.416179411 +0000 UTC m=+2182.921777893" watchObservedRunningTime="2026-07-06 20:00:01.418363414 +0000 UTC m=+2182.923961898" Jul 06 20:00:04.532885 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:04.532849 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:07.404034 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:07.404005 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:12.564612 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:12.564561 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:12.947027 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:12.946989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:13.055546 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.055511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:13.155869 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.155835 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:13.377004 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.376929 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 20:00:13.377155 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.377130 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" podUID="468b230d-56c4-467c-91ab-d8e8db633fcb" containerName="maas-api" containerID="cri-o://591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87" gracePeriod=30 Jul 06 20:00:13.402657 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.402628 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:13.630093 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.630020 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 20:00:13.800245 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.800213 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") pod \"468b230d-56c4-467c-91ab-d8e8db633fcb\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " Jul 06 20:00:13.800245 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.800253 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca\") pod \"468b230d-56c4-467c-91ab-d8e8db633fcb\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " Jul 06 20:00:13.800522 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.800420 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8hvx\" (UniqueName: \"kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx\") pod \"468b230d-56c4-467c-91ab-d8e8db633fcb\" (UID: \"468b230d-56c4-467c-91ab-d8e8db633fcb\") " Jul 06 20:00:13.800676 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.800654 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "468b230d-56c4-467c-91ab-d8e8db633fcb" (UID: "468b230d-56c4-467c-91ab-d8e8db633fcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:00:13.800764 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.800750 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468b230d-56c4-467c-91ab-d8e8db633fcb-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:13.802530 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.802510 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "468b230d-56c4-467c-91ab-d8e8db633fcb" (UID: "468b230d-56c4-467c-91ab-d8e8db633fcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:00:13.802652 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.802524 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx" (OuterVolumeSpecName: "kube-api-access-w8hvx") pod "468b230d-56c4-467c-91ab-d8e8db633fcb" (UID: "468b230d-56c4-467c-91ab-d8e8db633fcb"). InnerVolumeSpecName "kube-api-access-w8hvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:13.901434 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.901361 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468b230d-56c4-467c-91ab-d8e8db633fcb-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:13.901434 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:13.901388 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8hvx\" (UniqueName: \"kubernetes.io/projected/468b230d-56c4-467c-91ab-d8e8db633fcb-kube-api-access-w8hvx\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:14.458167 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.458128 2555 generic.go:356] "Generic (PLEG): container finished" podID="468b230d-56c4-467c-91ab-d8e8db633fcb" containerID="591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87" exitCode=0 Jul 06 20:00:14.458356 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.458177 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" Jul 06 20:00:14.458356 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.458214 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" event={"ID":"468b230d-56c4-467c-91ab-d8e8db633fcb","Type":"ContainerDied","Data":"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87"} Jul 06 20:00:14.458356 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.458270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54" event={"ID":"468b230d-56c4-467c-91ab-d8e8db633fcb","Type":"ContainerDied","Data":"cb5a54e1e651b0c8a5422494e9c09c79ca3efed1dc9b231cb96052c869d7a764"} Jul 06 20:00:14.458356 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.458307 2555 scope.go:117] "RemoveContainer" containerID="591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87" Jul 06 20:00:14.467111 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.467095 2555 scope.go:117] "RemoveContainer" containerID="591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87" Jul 06 20:00:14.467370 ip-10-0-133-207 kubenswrapper[2555]: E0706 20:00:14.467351 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87\": container with ID starting with 591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87 not found: ID does not exist" containerID="591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87" Jul 06 20:00:14.467444 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.467380 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87"} err="failed to get container status \"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87\": rpc error: code = NotFound desc = could not find container \"591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87\": container with ID starting with 591246d82d808f4553a49b68052f3c0ef76e30a8d9e5acfeb313356e4619be87 not found: ID does not exist" Jul 06 20:00:14.481223 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.481193 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 20:00:14.485382 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.485360 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c0c2c4-5c9bb5bf8b-zsq54"] Jul 06 20:00:14.967430 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:14.967401 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468b230d-56c4-467c-91ab-d8e8db633fcb" path="/var/lib/kubelet/pods/468b230d-56c4-467c-91ab-d8e8db633fcb/volumes" Jul 06 20:00:19.874934 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:19.874891 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:19.975294 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:19.975260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:20.256013 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.255979 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:20.406286 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.406249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:20.504487 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.504451 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:20.684091 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.683996 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:20.684268 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.684212 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" podUID="404c0867-fc6e-4105-9b3f-07fac496a3e6" containerName="maas-api" containerID="cri-o://1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2" gracePeriod=30 Jul 06 20:00:20.716219 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.716174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:20.941381 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.941313 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:20.968775 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.968744 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca\") pod \"404c0867-fc6e-4105-9b3f-07fac496a3e6\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " Jul 06 20:00:20.968940 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.968831 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q2x6\" (UniqueName: \"kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6\") pod \"404c0867-fc6e-4105-9b3f-07fac496a3e6\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " Jul 06 20:00:20.968940 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.968906 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls\") pod \"404c0867-fc6e-4105-9b3f-07fac496a3e6\" (UID: \"404c0867-fc6e-4105-9b3f-07fac496a3e6\") " Jul 06 20:00:20.969165 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.969123 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "404c0867-fc6e-4105-9b3f-07fac496a3e6" (UID: "404c0867-fc6e-4105-9b3f-07fac496a3e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:00:20.971151 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.971120 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "404c0867-fc6e-4105-9b3f-07fac496a3e6" (UID: "404c0867-fc6e-4105-9b3f-07fac496a3e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:00:20.971458 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:20.971440 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6" (OuterVolumeSpecName: "kube-api-access-7q2x6") pod "404c0867-fc6e-4105-9b3f-07fac496a3e6" (UID: "404c0867-fc6e-4105-9b3f-07fac496a3e6"). InnerVolumeSpecName "kube-api-access-7q2x6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:21.069955 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.069909 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404c0867-fc6e-4105-9b3f-07fac496a3e6-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:21.069955 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.069948 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q2x6\" (UniqueName: \"kubernetes.io/projected/404c0867-fc6e-4105-9b3f-07fac496a3e6-kube-api-access-7q2x6\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:21.069955 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.069958 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404c0867-fc6e-4105-9b3f-07fac496a3e6-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:00:21.493653 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.493619 2555 generic.go:356] "Generic (PLEG): container finished" podID="404c0867-fc6e-4105-9b3f-07fac496a3e6" containerID="1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2" exitCode=0 Jul 06 20:00:21.493838 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.493676 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" Jul 06 20:00:21.493838 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.493711 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" event={"ID":"404c0867-fc6e-4105-9b3f-07fac496a3e6","Type":"ContainerDied","Data":"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2"} Jul 06 20:00:21.493838 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.493767 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v" event={"ID":"404c0867-fc6e-4105-9b3f-07fac496a3e6","Type":"ContainerDied","Data":"5cac56b594de9aeffb027cbc82320b9d7b952536040425ae9d9720e95afb0831"} Jul 06 20:00:21.493838 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.493791 2555 scope.go:117] "RemoveContainer" containerID="1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2" Jul 06 20:00:21.503132 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.503108 2555 scope.go:117] "RemoveContainer" containerID="1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2" Jul 06 20:00:21.503402 ip-10-0-133-207 kubenswrapper[2555]: E0706 20:00:21.503384 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2\": container with ID starting with 1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2 not found: ID does not exist" containerID="1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2" Jul 06 20:00:21.503459 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.503409 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2"} err="failed to get container status \"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2\": rpc error: code = NotFound desc = could not find container \"1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2\": container with ID starting with 1f0e53a5af0bcc7459e27e5076efe837b9ec086f7d9f93d4168f1a01437b38c2 not found: ID does not exist" Jul 06 20:00:21.516999 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.516970 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:21.521928 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:21.521902 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bc0ca2-6455746755-kd55v"] Jul 06 20:00:22.967727 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:22.967696 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="404c0867-fc6e-4105-9b3f-07fac496a3e6" path="/var/lib/kubelet/pods/404c0867-fc6e-4105-9b3f-07fac496a3e6/volumes" Jul 06 20:00:27.367087 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:27.367057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:27.466064 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:27.466030 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:27.715450 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:27.715411 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:27.893108 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:27.893076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.185238 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.185198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.262621 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.262567 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:00:30.263480 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263452 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468b230d-56c4-467c-91ab-d8e8db633fcb" containerName="maas-api" Jul 06 20:00:30.263480 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263478 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="468b230d-56c4-467c-91ab-d8e8db633fcb" containerName="maas-api" Jul 06 20:00:30.263722 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263536 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404c0867-fc6e-4105-9b3f-07fac496a3e6" containerName="maas-api" Jul 06 20:00:30.263722 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263545 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="404c0867-fc6e-4105-9b3f-07fac496a3e6" containerName="maas-api" Jul 06 20:00:30.263722 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263669 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="404c0867-fc6e-4105-9b3f-07fac496a3e6" containerName="maas-api" Jul 06 20:00:30.263722 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.263688 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="468b230d-56c4-467c-91ab-d8e8db633fcb" containerName="maas-api" Jul 06 20:00:30.264177 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.264131 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.268631 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.268585 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.271453 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.271429 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-e1f716-serving-cert\"" Jul 06 20:00:30.278044 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.278021 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:00:30.327699 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.327654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.355685 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.355653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.355873 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.355719 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59c4s\" (UniqueName: \"kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.355873 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.355791 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.456555 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.456515 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.456772 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.456576 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59c4s\" (UniqueName: \"kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.456772 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.456619 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.456920 ip-10-0-133-207 kubenswrapper[2555]: E0706 20:00:30.456779 2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-e1f716-serving-cert: secret "maas-api-e2e-shared-a-e1f716-serving-cert" not found Jul 06 20:00:30.456920 ip-10-0-133-207 kubenswrapper[2555]: E0706 20:00:30.456863 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls podName:fef262fe-471d-49e6-b00d-68997ae2e485 nodeName:}" failed. No retries permitted until 2026-07-06 20:00:30.956841702 +0000 UTC m=+2212.462440182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls") pod "maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" (UID: "fef262fe-471d-49e6-b00d-68997ae2e485") : secret "maas-api-e2e-shared-a-e1f716-serving-cert" not found Jul 06 20:00:30.457355 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.457333 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.471068 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.471032 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59c4s\" (UniqueName: \"kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.632400 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.632329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.791543 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.791445 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.891374 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.891339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:30.962340 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.962301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:30.964999 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:30.964974 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") pod \"maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:31.021429 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:31.021392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:31.182436 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:31.182347 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:31.332073 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:31.332016 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:00:31.334632 ip-10-0-133-207 kubenswrapper[2555]: W0706 20:00:31.334581 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef262fe_471d_49e6_b00d_68997ae2e485.slice/crio-b0d9b93e5a790e8009230ceb0f6908053b687241f07e03f255dd68018e4167e8 WatchSource:0}: Error finding container b0d9b93e5a790e8009230ceb0f6908053b687241f07e03f255dd68018e4167e8: Status 404 returned error can't find the container with id b0d9b93e5a790e8009230ceb0f6908053b687241f07e03f255dd68018e4167e8 Jul 06 20:00:31.550583 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:31.550551 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" event={"ID":"fef262fe-471d-49e6-b00d-68997ae2e485","Type":"ContainerStarted","Data":"b0d9b93e5a790e8009230ceb0f6908053b687241f07e03f255dd68018e4167e8"} Jul 06 20:00:32.558000 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:32.557909 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" event={"ID":"fef262fe-471d-49e6-b00d-68997ae2e485","Type":"ContainerStarted","Data":"8a8ccb892222b6b248570d0441d4e77b1fda0cd8a8d1d9656bd93a1a903689bf"} Jul 06 20:00:32.558000 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:32.557969 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:32.579206 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:32.579150 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" podStartSLOduration=1.690172622 podStartE2EDuration="2.579135715s" totalImagesPullingTime="888.963093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:32.578381779 +0000 UTC m=+2214.083980263" watchObservedRunningTime="2026-07-06 20:00:32.579135715 +0000 UTC m=+2214.084734196" Jul 06 20:00:34.699144 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:34.699106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:35.016232 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:35.016132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:35.327367 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:35.327278 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:35.855443 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:35.855382 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.280685 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.280648 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.396369 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.396321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.398869 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.398839 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:00:38.403370 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.403350 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.406334 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.406305 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-035267-serving-cert\"" Jul 06 20:00:38.416486 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.416458 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:00:38.474402 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.474362 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.542183 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.542090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.542183 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.542132 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.542406 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.542199 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4f2\" (UniqueName: \"kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.568947 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.568913 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:00:38.572969 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.572950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.643196 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.643157 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.643391 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.643215 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.643391 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.643272 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4f2\" (UniqueName: \"kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.644048 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.643968 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.645750 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.645729 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.653323 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.653296 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4f2\" (UniqueName: \"kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2\") pod \"maas-api-e2e-shared-b-035267-864557b8bb-rtvl8\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.716130 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.716096 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:38.869713 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.869689 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:00:38.872107 ip-10-0-133-207 kubenswrapper[2555]: W0706 20:00:38.872081 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8258bc_db57_486b_a665_c81fc94cc986.slice/crio-9dcf0e90308e92d6e189368b5aaf0785d6ff00c54829574ab665a0ad0d98b4cc WatchSource:0}: Error finding container 9dcf0e90308e92d6e189368b5aaf0785d6ff00c54829574ab665a0ad0d98b4cc: Status 404 returned error can't find the container with id 9dcf0e90308e92d6e189368b5aaf0785d6ff00c54829574ab665a0ad0d98b4cc Jul 06 20:00:38.983904 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:38.983862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:39.121140 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:39.121109 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:39.595477 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:39.595442 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" event={"ID":"ee8258bc-db57-486b-a665-c81fc94cc986","Type":"ContainerStarted","Data":"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96"} Jul 06 20:00:39.595937 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:39.595500 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" event={"ID":"ee8258bc-db57-486b-a665-c81fc94cc986","Type":"ContainerStarted","Data":"9dcf0e90308e92d6e189368b5aaf0785d6ff00c54829574ab665a0ad0d98b4cc"} Jul 06 20:00:39.595937 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:39.595617 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:39.618190 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:39.618137 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" podStartSLOduration=1.0539523690000001 podStartE2EDuration="1.618120334s" totalImagesPullingTime="564.167965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:39.617709754 +0000 UTC m=+2221.123308262" watchObservedRunningTime="2026-07-06 20:00:39.618120334 +0000 UTC m=+2221.123718831" Jul 06 20:00:42.983345 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:42.983311 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:45.606155 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:45.606124 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:00:51.574646 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:51.574608 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:51.670426 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:00:51.670388 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:00.877657 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:01:00.877557 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:01.206695 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:01:01.206660 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:01.272082 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:01:01.272044 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:29.045634 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:01:29.045572 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:23.257601 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:02:23.257560 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:23.798461 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:23.798416 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:26.227142 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:26.227105 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:36.748922 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:36.748883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:37.646930 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:37.646888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:42.610249 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:42.610198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:42.714433 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:42.714402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:44.561183 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:03:44.561141 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:00.465776 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:00.465677 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:00.519246 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:00.519210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:04.345523 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:04.345488 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:42.988559 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:42.988515 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:43.867327 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:43.867293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:52.864469 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:04:52.864433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:15.654156 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:05:15.654118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:42.147392 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:05:42.147299 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:42.226071 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:05:42.226029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:52.648074 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:05:52.648026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:52.761053 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:05:52.760993 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:01.942243 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:01.942205 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:02.046705 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:02.046664 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:18.234445 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:18.234408 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:19.161667 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:19.161623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:20.232513 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:20.232473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:20.708348 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:20.708308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:20.884769 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:20.884732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:20.984796 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:20.984702 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:21.216516 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.216485 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:06:21.216747 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.216708 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" podUID="fef262fe-471d-49e6-b00d-68997ae2e485" containerName="maas-api" containerID="cri-o://8a8ccb892222b6b248570d0441d4e77b1fda0cd8a8d1d9656bd93a1a903689bf" gracePeriod=30 Jul 06 20:06:21.259684 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.259577 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:21.344699 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.344645 2555 generic.go:356] "Generic (PLEG): container finished" podID="fef262fe-471d-49e6-b00d-68997ae2e485" containerID="8a8ccb892222b6b248570d0441d4e77b1fda0cd8a8d1d9656bd93a1a903689bf" exitCode=0 Jul 06 20:06:21.344859 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.344726 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" event={"ID":"fef262fe-471d-49e6-b00d-68997ae2e485","Type":"ContainerDied","Data":"8a8ccb892222b6b248570d0441d4e77b1fda0cd8a8d1d9656bd93a1a903689bf"} Jul 06 20:06:21.474280 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.474253 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:06:21.594446 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.594344 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") pod \"fef262fe-471d-49e6-b00d-68997ae2e485\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " Jul 06 20:06:21.594446 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.594409 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca\") pod \"fef262fe-471d-49e6-b00d-68997ae2e485\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " Jul 06 20:06:21.594742 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.594455 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59c4s\" (UniqueName: \"kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s\") pod \"fef262fe-471d-49e6-b00d-68997ae2e485\" (UID: \"fef262fe-471d-49e6-b00d-68997ae2e485\") " Jul 06 20:06:21.594879 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.594855 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fef262fe-471d-49e6-b00d-68997ae2e485" (UID: "fef262fe-471d-49e6-b00d-68997ae2e485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:06:21.596759 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.596737 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fef262fe-471d-49e6-b00d-68997ae2e485" (UID: "fef262fe-471d-49e6-b00d-68997ae2e485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:06:21.596927 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.596788 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s" (OuterVolumeSpecName: "kube-api-access-59c4s") pod "fef262fe-471d-49e6-b00d-68997ae2e485" (UID: "fef262fe-471d-49e6-b00d-68997ae2e485"). InnerVolumeSpecName "kube-api-access-59c4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:06:21.696196 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.696159 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef262fe-471d-49e6-b00d-68997ae2e485-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:06:21.696196 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.696190 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef262fe-471d-49e6-b00d-68997ae2e485-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:06:21.696411 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:21.696217 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59c4s\" (UniqueName: \"kubernetes.io/projected/fef262fe-471d-49e6-b00d-68997ae2e485-kube-api-access-59c4s\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:06:22.351363 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.351335 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" Jul 06 20:06:22.351853 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.351329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s" event={"ID":"fef262fe-471d-49e6-b00d-68997ae2e485","Type":"ContainerDied","Data":"b0d9b93e5a790e8009230ceb0f6908053b687241f07e03f255dd68018e4167e8"} Jul 06 20:06:22.351853 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.351473 2555 scope.go:117] "RemoveContainer" containerID="8a8ccb892222b6b248570d0441d4e77b1fda0cd8a8d1d9656bd93a1a903689bf" Jul 06 20:06:22.382463 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.382434 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:06:22.384652 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.384629 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e1f716-6d89d64f46-rcf6s"] Jul 06 20:06:22.967572 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:22.967542 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef262fe-471d-49e6-b00d-68997ae2e485" path="/var/lib/kubelet/pods/fef262fe-471d-49e6-b00d-68997ae2e485/volumes" Jul 06 20:06:25.777875 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:06:25.777834 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:04.215413 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:04.215318 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:04.552422 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:04.552333 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:04.757052 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:04.757022 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:04.970666 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:04.970633 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:05.349769 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:05.349682 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:05.654241 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:05.654148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:05.758275 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:05.758233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:05.983046 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:05.983009 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:07:05.983258 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:05.983232 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" podUID="ee8258bc-db57-486b-a665-c81fc94cc986" containerName="maas-api" containerID="cri-o://3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96" gracePeriod=30 Jul 06 20:07:06.030815 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.030785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:06.245411 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.245337 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:07:06.332950 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.332911 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw4f2\" (UniqueName: \"kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2\") pod \"ee8258bc-db57-486b-a665-c81fc94cc986\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " Jul 06 20:07:06.332950 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.332954 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls\") pod \"ee8258bc-db57-486b-a665-c81fc94cc986\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " Jul 06 20:07:06.333165 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.332983 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca\") pod \"ee8258bc-db57-486b-a665-c81fc94cc986\" (UID: \"ee8258bc-db57-486b-a665-c81fc94cc986\") " Jul 06 20:07:06.333351 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.333316 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee8258bc-db57-486b-a665-c81fc94cc986" (UID: "ee8258bc-db57-486b-a665-c81fc94cc986"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:07:06.335226 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.335198 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee8258bc-db57-486b-a665-c81fc94cc986" (UID: "ee8258bc-db57-486b-a665-c81fc94cc986"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:07:06.335323 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.335302 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2" (OuterVolumeSpecName: "kube-api-access-tw4f2") pod "ee8258bc-db57-486b-a665-c81fc94cc986" (UID: "ee8258bc-db57-486b-a665-c81fc94cc986"). InnerVolumeSpecName "kube-api-access-tw4f2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:07:06.434309 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.434269 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw4f2\" (UniqueName: \"kubernetes.io/projected/ee8258bc-db57-486b-a665-c81fc94cc986-kube-api-access-tw4f2\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:07:06.434309 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.434302 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8258bc-db57-486b-a665-c81fc94cc986-maas-api-tls\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:07:06.434309 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.434310 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8258bc-db57-486b-a665-c81fc94cc986-openshift-service-ca\") on node \"ip-10-0-133-207.ec2.internal\" DevicePath \"\"" Jul 06 20:07:06.576199 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.576112 2555 generic.go:356] "Generic (PLEG): container finished" podID="ee8258bc-db57-486b-a665-c81fc94cc986" containerID="3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96" exitCode=0 Jul 06 20:07:06.576199 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.576163 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" Jul 06 20:07:06.576199 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.576180 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" event={"ID":"ee8258bc-db57-486b-a665-c81fc94cc986","Type":"ContainerDied","Data":"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96"} Jul 06 20:07:06.576412 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.576232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8" event={"ID":"ee8258bc-db57-486b-a665-c81fc94cc986","Type":"ContainerDied","Data":"9dcf0e90308e92d6e189368b5aaf0785d6ff00c54829574ab665a0ad0d98b4cc"} Jul 06 20:07:06.576412 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.576262 2555 scope.go:117] "RemoveContainer" containerID="3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96" Jul 06 20:07:06.585754 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.585733 2555 scope.go:117] "RemoveContainer" containerID="3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96" Jul 06 20:07:06.586029 ip-10-0-133-207 kubenswrapper[2555]: E0706 20:07:06.586010 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96\": container with ID starting with 3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96 not found: ID does not exist" containerID="3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96" Jul 06 20:07:06.586078 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.586042 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96"} err="failed to get container status \"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96\": rpc error: code = NotFound desc = could not find container \"3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96\": container with ID starting with 3716ab0db25cabb1547a7e512492549e51184dcf025573790ab54d24c0bdfa96 not found: ID does not exist" Jul 06 20:07:06.604526 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.604488 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:07:06.609407 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.609381 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-035267-864557b8bb-rtvl8"] Jul 06 20:07:06.968261 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:06.968224 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8258bc-db57-486b-a665-c81fc94cc986" path="/var/lib/kubelet/pods/ee8258bc-db57-486b-a665-c81fc94cc986/volumes" Jul 06 20:07:13.418759 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:13.418719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:13.483348 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:13.483309 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:13.739203 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:13.739163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:13.976129 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:13.976090 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:19.469561 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:19.469526 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76c64f7c7b-bp2k4_75a13851-17da-4dad-9174-d48f18ca1ebc/maas-api/0.log" Jul 06 20:07:19.851179 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:19.851094 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-746f49cdbd-qzxdk_4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4/manager/0.log" Jul 06 20:07:20.535424 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:20.535399 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fcb8c7879-ztzgd_83bcb5f0-4604-4dcb-af4c-ef9b9f219685/copy-manifests/0.log" Jul 06 20:07:20.548315 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:20.548283 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fcb8c7879-ztzgd_83bcb5f0-4604-4dcb-af4c-ef9b9f219685/manager/1.log" Jul 06 20:07:20.676580 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:20.676554 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cxrc9_eb9e0057-0bea-4a48-842d-9eae25716f12/postgres/0.log" Jul 06 20:07:23.529607 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:23.529538 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9kk6r_d490081a-11d5-45c5-a3b3-539a58963421/discovery/0.log" Jul 06 20:07:24.031096 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:24.031056 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mx8xg_6e0a1082-94ab-48fd-bc75-b77e871f4f2c/payload-pre-processing/0.log" Jul 06 20:07:24.177092 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:24.177058 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dwns9_a3d6c60b-a840-4b99-b709-aef3dba23b81/payload-processing/0.log" Jul 06 20:07:24.310990 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:24.310906 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74779b8599-gtvkx_94cabef5-e90c-44fe-aa25-41672255ad22/router/0.log" Jul 06 20:07:24.663926 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:24.663834 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-r6k7k_a361e235-5a1c-49f2-a00b-f7193add49d1/main/0.log" Jul 06 20:07:24.902904 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:24.902871 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5lsn8_545c838a-8515-48b5-961b-5fddc038acea/main/0.log" Jul 06 20:07:25.026206 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:25.026179 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsxqrm_1c8b1c25-5c32-40b5-b697-649753a42c32/main/0.log" Jul 06 20:07:29.065504 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:29.065464 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:30.583267 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:30.583227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:30.863221 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:30.863134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:32.323034 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:32.323001 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ff8dv_afc2758e-8980-4335-8c35-6453b75a191e/global-pull-secret-syncer/0.log" Jul 06 20:07:32.436892 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:32.436866 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9dhl7_1e0344a1-a744-4dc7-b794-257feb4ebc13/konnectivity-agent/0.log" Jul 06 20:07:32.600227 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:32.600147 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-207.ec2.internal_66f1f7c007b6c83ee623a99644ad20e8/haproxy/0.log" Jul 06 20:07:39.730115 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.730078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/alertmanager/0.log" Jul 06 20:07:39.770227 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.770194 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/config-reloader/0.log" Jul 06 20:07:39.803607 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.803573 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/kube-rbac-proxy-web/0.log" Jul 06 20:07:39.840741 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.840715 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/kube-rbac-proxy/0.log" Jul 06 20:07:39.866210 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.866184 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/kube-rbac-proxy-metric/0.log" Jul 06 20:07:39.891967 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.891885 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/prom-label-proxy/0.log" Jul 06 20:07:39.921639 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.921616 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_026bde0f-1fa4-4925-9ac4-c1955a16c0c7/init-config-reloader/0.log" Jul 06 20:07:39.998117 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:39.998078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-n2j8t_2033a09b-cb17-417a-b84c-1e1f7eb5f06f/cluster-monitoring-operator/0.log" Jul 06 20:07:40.047340 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.047306 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-86k7p_6befcbd0-7eb2-498e-8d6f-df9177fee622/kube-state-metrics/0.log" Jul 06 20:07:40.086520 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.086491 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-86k7p_6befcbd0-7eb2-498e-8d6f-df9177fee622/kube-rbac-proxy-main/0.log" Jul 06 20:07:40.119051 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.119024 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-86k7p_6befcbd0-7eb2-498e-8d6f-df9177fee622/kube-rbac-proxy-self/0.log" Jul 06 20:07:40.227338 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.227308 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-7vt7b_f9e7d964-7d87-4b1f-83e3-e0a9f05fe156/monitoring-plugin/0.log" Jul 06 20:07:40.408258 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.408230 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwfnc_22026970-8d60-42d1-9332-6cf85027cd95/node-exporter/0.log" Jul 06 20:07:40.471719 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.471684 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwfnc_22026970-8d60-42d1-9332-6cf85027cd95/kube-rbac-proxy/0.log" Jul 06 20:07:40.510706 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.510626 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwfnc_22026970-8d60-42d1-9332-6cf85027cd95/init-textfile/0.log" Jul 06 20:07:40.787578 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.787493 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/prometheus/0.log" Jul 06 20:07:40.818706 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.818672 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/config-reloader/0.log" Jul 06 20:07:40.850165 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.850138 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/thanos-sidecar/0.log" Jul 06 20:07:40.881227 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.881200 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/kube-rbac-proxy-web/0.log" Jul 06 20:07:40.917540 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.917507 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/kube-rbac-proxy/0.log" Jul 06 20:07:40.955810 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.955778 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/kube-rbac-proxy-thanos/0.log" Jul 06 20:07:40.987453 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:40.987422 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd4a7fe8-4ea1-471f-9218-507e7f9e58b9/init-config-reloader/0.log" Jul 06 20:07:41.043147 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.043061 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vmqlb_07bdcef8-1b82-4acb-a045-2d540409f00a/prometheus-operator/0.log" Jul 06 20:07:41.077421 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.077388 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vmqlb_07bdcef8-1b82-4acb-a045-2d540409f00a/kube-rbac-proxy/0.log" Jul 06 20:07:41.259322 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.259291 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/thanos-query/0.log" Jul 06 20:07:41.286939 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.286910 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/kube-rbac-proxy-web/0.log" Jul 06 20:07:41.320358 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.320288 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/kube-rbac-proxy/0.log" Jul 06 20:07:41.352292 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.352265 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/prom-label-proxy/0.log" Jul 06 20:07:41.389880 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.389846 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/kube-rbac-proxy-rules/0.log" Jul 06 20:07:41.424899 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.424868 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c7c865454-hxnnh_c381d033-1248-471e-b89b-edfd4a7c490b/kube-rbac-proxy-metrics/0.log" Jul 06 20:07:41.855309 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855272 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8"] Jul 06 20:07:41.855751 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855703 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef262fe-471d-49e6-b00d-68997ae2e485" containerName="maas-api" Jul 06 20:07:41.855751 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855714 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef262fe-471d-49e6-b00d-68997ae2e485" containerName="maas-api" Jul 06 20:07:41.855751 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855729 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8258bc-db57-486b-a665-c81fc94cc986" containerName="maas-api" Jul 06 20:07:41.855751 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855735 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8258bc-db57-486b-a665-c81fc94cc986" containerName="maas-api" Jul 06 20:07:41.855905 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855796 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8258bc-db57-486b-a665-c81fc94cc986" containerName="maas-api" Jul 06 20:07:41.855905 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.855805 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef262fe-471d-49e6-b00d-68997ae2e485" containerName="maas-api" Jul 06 20:07:41.860719 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.860697 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.863859 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.863841 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7tz4\"/\"openshift-service-ca.crt\"" Jul 06 20:07:41.863998 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.863875 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7tz4\"/\"kube-root-ca.crt\"" Jul 06 20:07:41.865179 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.865162 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c7tz4\"/\"default-dockercfg-ph479\"" Jul 06 20:07:41.874781 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.874759 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8"] Jul 06 20:07:41.882238 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.882215 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-proc\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.882343 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.882273 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-lib-modules\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.882343 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.882302 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkgj\" (UniqueName: \"kubernetes.io/projected/bff411d1-c3cb-4758-999a-908355e77838-kube-api-access-npkgj\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.882458 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.882395 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-podres\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.882517 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.882458 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-sys\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984208 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-proc\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984381 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-lib-modules\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984381 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkgj\" (UniqueName: \"kubernetes.io/projected/bff411d1-c3cb-4758-999a-908355e77838-kube-api-access-npkgj\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984381 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-podres\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984560 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-sys\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984646 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984572 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-sys\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984712 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984681 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-proc\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.984837 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.984817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-lib-modules\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:41.990377 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:41.990346 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bff411d1-c3cb-4758-999a-908355e77838-podres\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:42.004487 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.004457 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkgj\" (UniqueName: \"kubernetes.io/projected/bff411d1-c3cb-4758-999a-908355e77838-kube-api-access-npkgj\") pod \"perf-node-gather-daemonset-khqc8\" (UID: \"bff411d1-c3cb-4758-999a-908355e77838\") " pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:42.171095 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.170991 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:42.314999 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.314975 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8"] Jul 06 20:07:42.317646 ip-10-0-133-207 kubenswrapper[2555]: W0706 20:07:42.317616 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbff411d1_c3cb_4758_999a_908355e77838.slice/crio-5f8789212d02595ac2f042d1792ad264b1f83e9caaaf31b5c8b2e517176f281a WatchSource:0}: Error finding container 5f8789212d02595ac2f042d1792ad264b1f83e9caaaf31b5c8b2e517176f281a: Status 404 returned error can't find the container with id 5f8789212d02595ac2f042d1792ad264b1f83e9caaaf31b5c8b2e517176f281a Jul 06 20:07:42.756907 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.756871 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" event={"ID":"bff411d1-c3cb-4758-999a-908355e77838","Type":"ContainerStarted","Data":"82de6d74be147e9beb36e372086c7b263fc919695843b8e5bd94c3a604404e00"} Jul 06 20:07:42.757102 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.756922 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" event={"ID":"bff411d1-c3cb-4758-999a-908355e77838","Type":"ContainerStarted","Data":"5f8789212d02595ac2f042d1792ad264b1f83e9caaaf31b5c8b2e517176f281a"} Jul 06 20:07:42.757102 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.757002 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:42.777053 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:42.776998 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" podStartSLOduration=1.7769811500000001 podStartE2EDuration="1.77698115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:42.776870321 +0000 UTC m=+2644.282468801" watchObservedRunningTime="2026-07-06 20:07:42.77698115 +0000 UTC m=+2644.282579633" Jul 06 20:07:44.000127 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:44.000101 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c4f849554-v8nlx_90c3d902-6cd5-4767-9cb6-83cd1c7cc8b4/console/0.log" Jul 06 20:07:44.703938 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:44.703906 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-5j2bh_85994fa4-271e-41ea-a0e3-5048ff7ed130/volume-data-source-validator/0.log" Jul 06 20:07:46.009100 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:46.009067 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llznx_85e87373-0d72-46c5-ae18-6c4af955b811/dns/0.log" Jul 06 20:07:46.039699 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:46.039675 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llznx_85e87373-0d72-46c5-ae18-6c4af955b811/kube-rbac-proxy/0.log" Jul 06 20:07:46.167754 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:46.167722 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rdrrh_d8c2649c-87a9-47ab-9ae1-1b68ae762cbe/dns-node-resolver/0.log" Jul 06 20:07:46.920934 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:46.920900 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dd5x_8dc81b78-949e-445a-adb7-cbd3e162034f/node-ca/0.log" Jul 06 20:07:48.285985 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:48.285946 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9kk6r_d490081a-11d5-45c5-a3b3-539a58963421/discovery/0.log" Jul 06 20:07:48.459871 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:48.459839 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mx8xg_6e0a1082-94ab-48fd-bc75-b77e871f4f2c/payload-pre-processing/0.log" Jul 06 20:07:48.540866 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:48.540780 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dwns9_a3d6c60b-a840-4b99-b709-aef3dba23b81/payload-processing/0.log" Jul 06 20:07:48.565986 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:48.565957 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74779b8599-gtvkx_94cabef5-e90c-44fe-aa25-41672255ad22/router/0.log" Jul 06 20:07:48.772318 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:48.772290 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c7tz4/perf-node-gather-daemonset-khqc8" Jul 06 20:07:49.549726 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:49.549694 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5hzq_d50d1397-7512-454f-9583-a84e4a7b26d4/serve-healthcheck-canary/0.log" Jul 06 20:07:50.468306 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:50.468273 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kll4p_7e94f7ba-3a50-4f14-bd09-b4ff52d246c1/kube-rbac-proxy/0.log" Jul 06 20:07:50.499943 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:50.499909 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kll4p_7e94f7ba-3a50-4f14-bd09-b4ff52d246c1/exporter/0.log" Jul 06 20:07:50.530775 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:50.530741 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kll4p_7e94f7ba-3a50-4f14-bd09-b4ff52d246c1/extractor/0.log" Jul 06 20:07:52.367952 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:52.367923 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9kk6r_d490081a-11d5-45c5-a3b3-539a58963421/discovery/0.log" Jul 06 20:07:52.504309 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:52.504273 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mx8xg_6e0a1082-94ab-48fd-bc75-b77e871f4f2c/payload-pre-processing/0.log" Jul 06 20:07:52.565986 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:52.565954 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dwns9_a3d6c60b-a840-4b99-b709-aef3dba23b81/payload-processing/0.log" Jul 06 20:07:52.588884 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:52.588846 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74779b8599-gtvkx_94cabef5-e90c-44fe-aa25-41672255ad22/router/0.log" Jul 06 20:07:53.440225 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:53.440193 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76c64f7c7b-bp2k4_75a13851-17da-4dad-9174-d48f18ca1ebc/maas-api/0.log" Jul 06 20:07:53.682435 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:53.682404 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-746f49cdbd-qzxdk_4ab7fe55-ec24-4cbe-8aa5-4e658cde87f4/manager/0.log" Jul 06 20:07:53.954603 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:53.954569 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9kk6r_d490081a-11d5-45c5-a3b3-539a58963421/discovery/0.log" Jul 06 20:07:54.174765 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.174667 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fcb8c7879-ztzgd_83bcb5f0-4604-4dcb-af4c-ef9b9f219685/manager/1.log" Jul 06 20:07:54.177968 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.177942 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mx8xg_6e0a1082-94ab-48fd-bc75-b77e871f4f2c/payload-pre-processing/0.log" Jul 06 20:07:54.221857 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.221825 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fcb8c7879-ztzgd_83bcb5f0-4604-4dcb-af4c-ef9b9f219685/manager/0.log" Jul 06 20:07:54.243051 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.243025 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fcb8c7879-ztzgd_83bcb5f0-4604-4dcb-af4c-ef9b9f219685/copy-manifests/0.log" Jul 06 20:07:54.253999 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.253967 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dwns9_a3d6c60b-a840-4b99-b709-aef3dba23b81/payload-processing/0.log" Jul 06 20:07:54.272901 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.272878 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cxrc9_eb9e0057-0bea-4a48-842d-9eae25716f12/postgres/0.log" Jul 06 20:07:54.273167 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:54.273144 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74779b8599-gtvkx_94cabef5-e90c-44fe-aa25-41672255ad22/router/0.log" Jul 06 20:07:55.674943 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:55.674914 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cf88bd784-wb2lg_a29180a4-64ad-4de7-8080-8b974bed1a5f/manager/1.log" Jul 06 20:07:55.677033 ip-10-0-133-207 kubenswrapper[2555]: I0706 20:07:55.677000 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cf88bd784-wb2lg_a29180a4-64ad-4de7-8080-8b974bed1a5f/manager/0.log"