Jul 09 14:00:31.405010 ip-10-0-140-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 14:00:31.405021 ip-10-0-140-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 14:00:31.405028 ip-10-0-140-134 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 14:00:31.405270 ip-10-0-140-134 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 14:00:41.503248 ip-10-0-140-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 14:00:41.503265 ip-10-0-140-134 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 623d9f6f10d94f73a763f39701f6c04b -- Jul 09 14:02:16.991147 ip-10-0-140-134 systemd[1]: Starting Kubernetes Kubelet... Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.448718 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455453 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455464 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455467 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455470 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455473 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455476 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455479 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.465997 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455483 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455486 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455489 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455491 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455494 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455496 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455499 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455501 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455504 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455507 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455513 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455515 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455518 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455520 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455523 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455525 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455528 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455531 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455533 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.530833 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455538 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455541 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455545 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455548 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455551 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455554 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455557 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455560 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455563 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455565 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455568 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455570 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455573 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455575 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455577 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455580 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455583 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455586 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455588 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455591 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.535276 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455593 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.525673 ip-10-0-140-134 systemd[1]: Started Kubernetes Kubelet. Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455595 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455598 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455601 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455605 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455608 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455611 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455614 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455617 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455621 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455624 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455627 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455629 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455632 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455635 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455637 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455640 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455642 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455645 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.539059 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455647 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455650 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455652 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455655 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455657 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455661 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455663 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455666 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455669 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455671 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455675 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455678 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455680 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455683 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455686 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455688 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455691 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455693 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455695 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455698 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.542938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455701 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455703 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455706 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455708 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455711 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455713 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455716 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455718 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455720 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455723 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455725 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455728 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455730 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.455732 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456420 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456427 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456430 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456433 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456436 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456439 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.546579 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456441 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456444 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456447 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456453 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456456 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456459 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456461 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456464 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456466 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456469 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456472 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456474 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456477 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456479 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456482 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456484 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456487 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456489 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456492 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456494 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.550913 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456497 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456500 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456503 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456506 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456510 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456513 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456516 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456519 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456524 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456526 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456529 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456532 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456534 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456536 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456539 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456542 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456545 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456548 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456550 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456552 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.554664 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456555 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456557 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456560 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456562 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456565 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456568 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456572 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456575 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456579 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456582 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456585 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456592 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456594 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456597 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456600 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456604 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456607 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456610 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456613 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.558747 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456615 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456617 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456620 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456622 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456625 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456627 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456630 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456633 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456635 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456638 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456641 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456643 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456646 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456648 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456650 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456653 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456656 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456658 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456661 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456664 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.563151 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456666 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456669 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456671 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456674 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456677 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456680 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456682 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456685 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456688 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456691 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456693 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456696 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456698 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.456700 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457098 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457104 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457107 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457110 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.717294 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457113 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457116 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457118 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457121 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457124 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457127 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457130 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457132 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457135 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457137 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457140 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457142 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457145 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457147 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457150 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457152 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457154 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457157 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457161 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457164 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.720205 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457166 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457168 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457172 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457175 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457177 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457179 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457182 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457184 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457187 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457189 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457192 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457194 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457197 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457201 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457204 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457206 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457209 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457211 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457214 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.723705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457216 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457219 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457221 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457224 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457227 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457230 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457232 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457236 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457239 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457242 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457245 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457249 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457252 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457254 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457256 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457260 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457263 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457265 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457267 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.726263 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457270 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457273 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457275 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457278 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457280 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457282 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457285 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457287 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457290 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457292 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457294 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457297 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457300 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457303 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457305 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457308 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457310 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457313 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457315 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457317 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.729375 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457319 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457322 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457324 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457326 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457330 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457332 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457335 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457337 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457341 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457343 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457345 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457349 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457352 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457355 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457358 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457361 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.457363 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457423 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457429 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457435 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457439 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 14:02:17.729950 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457443 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457446 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457449 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457453 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457456 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457460 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457463 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457466 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457469 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457472 2576 flags.go:64] FLAG: --cgroup-root="" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457474 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457477 2576 flags.go:64] FLAG: --client-ca-file="" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457480 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457483 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457487 2576 flags.go:64] FLAG: --cluster-domain="" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457491 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457494 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457497 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457500 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457511 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457514 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457517 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457520 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457523 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 09 14:02:17.730676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457526 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457529 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457531 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457534 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457538 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457541 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457544 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457547 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457549 2576 flags.go:64] FLAG: --enable-server="true" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457552 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457556 2576 flags.go:64] FLAG: --event-burst="100" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457559 2576 flags.go:64] FLAG: --event-qps="50" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457562 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457565 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457568 2576 flags.go:64] FLAG: --eviction-hard="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457572 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457575 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457578 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457580 2576 flags.go:64] FLAG: --eviction-soft="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457583 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457587 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457589 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457592 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457596 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457598 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 14:02:17.731422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457601 2576 flags.go:64] FLAG: --feature-gates="" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457605 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457609 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457612 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457615 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457618 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457621 2576 flags.go:64] FLAG: --help="false" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457623 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457626 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457629 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457631 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457635 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457638 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457641 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457643 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457646 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457649 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457651 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457654 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457657 2576 flags.go:64] FLAG: --kube-reserved="" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457660 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457663 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457666 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457669 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 14:02:17.732524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457671 2576 flags.go:64] FLAG: --lock-file="" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457674 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457676 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457679 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457684 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457687 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457691 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457693 2576 flags.go:64] FLAG: --logging-format="text" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457696 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457700 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457703 2576 flags.go:64] FLAG: --manifest-url="" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457706 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457710 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457713 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457717 2576 flags.go:64] FLAG: --max-pods="110" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457719 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457722 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457725 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457728 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457731 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457733 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457736 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457743 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457745 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457748 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 14:02:17.734142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457751 2576 flags.go:64] FLAG: --pod-cidr="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457754 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457759 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457762 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457765 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457769 2576 flags.go:64] FLAG: --port="10250" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457772 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457775 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09f2f5a304aa967f1" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457778 2576 flags.go:64] FLAG: --qos-reserved="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457780 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457783 2576 flags.go:64] FLAG: --register-node="true" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457786 2576 flags.go:64] FLAG: --register-with-taints="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457789 2576 flags.go:64] FLAG: --registry-burst="10" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457792 2576 flags.go:64] FLAG: --registry-qps="5" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457796 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457798 2576 flags.go:64] FLAG: --reserved-memory="" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457801 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457806 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457809 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457811 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457814 2576 flags.go:64] FLAG: --runonce="false" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457817 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457820 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457822 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457825 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 14:02:17.735777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457828 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457830 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457833 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457836 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457838 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457841 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457844 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457846 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457850 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457852 2576 flags.go:64] FLAG: --system-cgroups="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457855 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457860 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457862 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457866 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457869 2576 flags.go:64] FLAG: --tls-min-version="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457872 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457874 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457889 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457892 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457895 2576 flags.go:64] FLAG: --v="2" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457899 2576 flags.go:64] FLAG: --version="false" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457904 2576 flags.go:64] FLAG: --vmodule="" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457907 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.457910 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458019 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.746534 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458022 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458025 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458028 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458030 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458033 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458035 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458038 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458040 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458043 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458045 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458048 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458051 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458053 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458056 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458058 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458060 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458063 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458065 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458068 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.747290 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458070 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458073 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458078 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458081 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458083 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458086 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458088 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458091 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458093 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458097 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458099 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458102 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458106 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458109 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458111 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458114 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458116 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458119 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458121 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.749096 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458124 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458127 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458129 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458132 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458134 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458137 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458139 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458141 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458144 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458146 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458149 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458151 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458153 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458156 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458158 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458160 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458165 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458168 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458172 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458174 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.749803 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458177 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458180 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458183 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458186 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458188 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458192 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458195 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458197 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458199 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458202 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458205 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458207 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458209 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458212 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458215 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458217 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458221 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458225 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458227 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.750995 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458230 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458232 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458235 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458237 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458240 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458242 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458245 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458247 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458250 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458252 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458255 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458257 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458260 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458262 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458265 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458270 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458273 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458275 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458279 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458282 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.751705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.458284 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.458290 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.466243 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.466256 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.466268 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.466273 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466325 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466329 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466333 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466337 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466340 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466343 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466346 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466349 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466351 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.756464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466354 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466358 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466362 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466364 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466367 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466370 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466372 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466376 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466378 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466381 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466384 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466387 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466390 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466392 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466397 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466400 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466403 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466407 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.757108 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466410 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466413 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466416 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466419 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466422 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466425 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466428 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466431 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466433 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466436 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466439 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466442 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466444 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466447 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466450 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466452 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466455 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466457 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466460 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466462 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.758340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466464 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466467 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466470 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466472 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466476 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466478 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466481 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466484 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466486 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466489 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466491 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466494 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466496 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466499 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466502 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466504 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466506 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466509 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466512 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466515 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.759155 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466518 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466520 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466522 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466525 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466527 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466530 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466532 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466535 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466537 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466540 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466542 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466545 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466547 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466549 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466552 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466555 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466558 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466563 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466565 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466568 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.759983 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466570 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466573 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466575 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466577 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466580 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466583 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466585 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466587 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466590 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466592 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466595 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466597 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.466602 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466707 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:02:17.851561 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466711 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466714 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466717 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466720 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466723 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466727 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466729 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466732 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466735 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466738 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466740 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466743 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466745 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466747 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466753 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466757 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466760 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466762 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466765 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466768 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:02:17.852244 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466770 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466773 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466775 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466778 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466780 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466782 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466785 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466787 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466790 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466792 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466795 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466797 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466801 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466804 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466807 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466810 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466813 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466816 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466819 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:02:17.853310 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466822 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466824 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466829 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466832 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466835 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466838 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466840 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466843 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466847 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466849 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466852 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466854 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466857 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466859 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466862 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466864 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466867 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466869 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466872 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466874 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:02:17.855433 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466891 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466896 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466899 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466901 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466904 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466906 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466908 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466911 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466913 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466917 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466921 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466923 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466926 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466929 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466933 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466935 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466938 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466940 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466943 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466946 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:02:17.856142 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466950 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466953 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466956 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466959 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466961 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466964 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466966 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466969 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466971 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466974 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466976 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466979 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466981 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466983 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466986 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466988 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466991 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466993 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:02:17.856830 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:17.466995 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.467000 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.467084 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.469482 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.471250 2576 server.go:1019] "Starting client certificate rotation" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.471343 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.471379 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.474129 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.478198 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.491068 2576 log.go:25] "Validated CRI v1 runtime API" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.493366 2576 log.go:25] "Validated CRI v1 image API" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.494136 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 14:02:17.859310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.496792 2576 fs.go:135] Filesystem UUIDs: map[418ffea1-ea03-410b-a924-ec9f3d2fcd60:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ff5216b1-360d-4440-8405-d62e3891e86f:/dev/nvme0n1p4] Jul 09 14:02:17.860024 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.496809 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 14:02:17.860024 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.503141 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.503159 2576 manager.go:216] Machine: {Timestamp:2026-07-09 14:02:17.501553125 +0000 UTC m=+0.379578539 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199650 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec282bb7f098fd66c86ea35adbfe86bf SystemUUID:ec282bb7-f098-fd66-c86e-a35adbfe86bf BootID:623d9f6f-10d9-4f73-a763-f39701f6c04b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7d:a4:21:fb:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7d:a4:21:fb:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:6b:9a:f4:64:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.503288 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.503412 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.506990 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.507013 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.507828 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.507837 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.507844 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.508565 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.510475 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.514996 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515107 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515134 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515148 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515163 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515171 2576 kubelet.go:398] "Adding apiserver pod source" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515178 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 14:02:17.861772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515440 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.515453 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.517506 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520003 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520037 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520247 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520261 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520266 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520272 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520279 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520306 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520872 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520895 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520901 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520906 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520930 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.520938 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.521770 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.521777 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.524953 2576 server.go:1268] "Started kubelet" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.525040 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.525103 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.525111 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.528772 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.529525 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.529561 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.529564 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.530406 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.533293 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.533207 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-134.ec2.internal.18c0a3f7e4858e86 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-134.ec2.internal,UID:ip-10-0-140-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-134.ec2.internal,},FirstTimestamp:2026-07-09 14:02:17.524924038 +0000 UTC m=+0.402949426,LastTimestamp:2026-07-09 14:02:17.524924038 +0000 UTC m=+0.402949426,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-134.ec2.internal,}" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.540296 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.540300 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.540355 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.540990 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541012 2576 factory.go:55] Registering systemd factory Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541022 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541035 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541026 2576 factory.go:223] Registration of the systemd container factory successfully Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541142 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541148 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541261 2576 factory.go:153] Registering CRI-O factory Jul 09 14:02:17.863545 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541271 2576 factory.go:223] Registration of the crio container factory successfully Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541311 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541329 2576 factory.go:103] Registering Raw factory Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541340 2576 manager.go:1196] Started watching for new ooms in manager Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.541629 2576 manager.go:349] Starting recovery of all containers Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.549112 2576 manager.go:354] Recovery completed Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.551085 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.552354 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjgh8" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.553465 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.554858 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555188 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555212 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555225 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555735 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555746 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555748 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.555762 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.557981 2576 policy_none.go:49] "None policy: Start" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.557994 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.558004 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.558913 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.558942 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.558956 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.560013 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjgh8" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.560320 2576 policy_none.go:47] "Start" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.564233 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-134.ec2.internal.18c0a3f7e6538a06 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-134.ec2.internal,UID:ip-10-0-140-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-134.ec2.internal,},FirstTimestamp:2026-07-09 14:02:17.555200518 +0000 UTC m=+0.433225911,LastTimestamp:2026-07-09 14:02:17.555200518 +0000 UTC m=+0.433225911,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-134.ec2.internal,}" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587420 2576 manager.go:337] "Starting Device Plugin manager" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.587446 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587455 2576 server.go:87] "Starting device plugin registration server" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587692 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587701 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587794 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587868 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.587893 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.588254 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.588305 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.647727 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.648784 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.648816 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 14:02:17.864964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.648834 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.648888 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.651647 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.687932 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.688630 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.688647 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.688657 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.688678 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.698436 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.698452 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-134.ec2.internal\": node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.735484 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.749746 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"] Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.749805 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.750559 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.750585 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.750595 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.752760 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.753657 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.753682 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.753694 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.753918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.754343 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.755333 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.755355 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.755364 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.755792 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.755810 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.756409 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.756432 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.756442 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.836217 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.843545 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.843568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.866018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.843583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46bed090be62d9ef320eca39a1d183b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"f46bed090be62d9ef320eca39a1d183b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:17.936971 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944288 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46bed090be62d9ef320eca39a1d183b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"f46bed090be62d9ef320eca39a1d183b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46bed090be62d9ef320eca39a1d183b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"f46bed090be62d9ef320eca39a1d183b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944355 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:17.960592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:17.944365 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831e3dd915e470f53b8a8ad37f2ef182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"831e3dd915e470f53b8a8ad37f2ef182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:18.059073 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.037663 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:18.163281 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.138128 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:18.273481 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.238558 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:18.376660 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.338968 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.395605 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.397390 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:18.400132 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.402915 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.414535 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.440250 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.440433 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.447681 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.453175 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.454011 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.462650 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.470748 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.470834 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.470876 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.470894 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:02:19.736655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:18.470873 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.755379 2576 apiserver.go:52] "Watching apiserver" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.773294 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.773584 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-h96rp","openshift-ovn-kubernetes/ovnkube-node-lmg5b","kube-system/global-pull-secret-syncer-8q4t5","kube-system/konnectivity-agent-qhggs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl","openshift-image-registry/node-ca-gncrb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","openshift-multus/multus-additional-cni-plugins-zfvtf","openshift-multus/multus-ljtkx","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal","openshift-cluster-node-tuning-operator/tuned-56knf","openshift-dns/node-resolver-c4r7k","openshift-multus/network-metrics-daemon-tvvs6","openshift-network-diagnostics/network-check-target-dpkkq"] Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.775882 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.777986 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.778262 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.778808 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.779048 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.779144 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6nwm6\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.780224 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.780454 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.780863 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8vjht\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.781550 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.781816 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.781937 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.781989 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.782082 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.782169 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.784006 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.784711 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.784981 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.785009 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c79z5\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786105 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.786183 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7ca6750-e97a-4417-bce0-567a03402717-ovn-node-metrics-cert\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786422 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e27433c-2659-4dc2-9405-b9f3851d935a-agent-certs\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786430 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.849060 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786443 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786443 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-kubelet\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786520 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-systemd-units\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-slash\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-node-log\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-log-socket\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-74bqg\"" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62c48dea-d112-4f60-8d97-5d1be6f76d5d-serviceca\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-systemd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786690 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-var-lib-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-bin\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786766 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-config\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786783 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-env-overrides\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786803 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jwqp\" (UniqueName: \"kubernetes.io/projected/b7ca6750-e97a-4417-bce0-567a03402717-kube-api-access-9jwqp\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.951131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e27433c-2659-4dc2-9405-b9f3851d935a-konnectivity-ca\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62c48dea-d112-4f60-8d97-5d1be6f76d5d-host\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-netns\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-etc-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-ovn\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-netd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786918 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-script-lib\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.786934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmxg7\" (UniqueName: \"kubernetes.io/projected/62c48dea-d112-4f60-8d97-5d1be6f76d5d-kube-api-access-mmxg7\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.788297 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.789977 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.790591 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.790598 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.790602 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.790958 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.791028 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plsbj\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.792689 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.792968 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.793060 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.793091 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.793201 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gxjfm\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.793328 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.793795 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.794775 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.794906 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.795067 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kz2hh\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.797092 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.797388 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rl7lj\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.797394 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.797687 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.952301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.799273 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.799384 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.799463 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.799474 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-txn5h\"" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.799518 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.801641 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.801682 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.803984 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 13:57:18 +0000 UTC" deadline="2028-04-06 10:09:01.23176007 +0000 UTC" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.804020 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15284h6m41.427742087s" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.809897 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bw22n" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.821448 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bw22n" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.882456 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-tuned\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887272 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887289 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-script-lib\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887322 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-systemd\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l874b\" (UniqueName: \"kubernetes.io/projected/87cb950f-3838-4997-a2a4-d85b4291d420-kube-api-access-l874b\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-system-cni-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-os-release\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmxg7\" (UniqueName: \"kubernetes.io/projected/62c48dea-d112-4f60-8d97-5d1be6f76d5d-kube-api-access-mmxg7\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.953079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7ca6750-e97a-4417-bce0-567a03402717-ovn-node-metrics-cert\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887552 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-kubelet-config\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887604 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysconfig\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887620 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-registration-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-config\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887651 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-kubelet\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-log-socket\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887724 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887730 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-kubelet\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887766 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-log-socket\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-modprobe-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887806 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q27lc\" (UniqueName: \"kubernetes.io/projected/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-kube-api-access-q27lc\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:19.954638 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-sys-fs\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-kubernetes\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887893 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h24dj\" (UniqueName: \"kubernetes.io/projected/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kube-api-access-h24dj\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-bin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887966 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-multus\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.887988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62c48dea-d112-4f60-8d97-5d1be6f76d5d-host\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888020 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-cnibin\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888034 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-os-release\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-hostroot\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-multus-certs\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888082 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62c48dea-d112-4f60-8d97-5d1be6f76d5d-host\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888095 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-etc-kubernetes\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-dbus\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-iptables-alerter-script\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-host-slash\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.955444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-cni-binary-copy\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888186 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87cb950f-3838-4997-a2a4-d85b4291d420-tmp-dir\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-socket-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-node-log\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888257 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888266 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-node-log\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888271 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-run\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-socket-dir-parent\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888295 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62c48dea-d112-4f60-8d97-5d1be6f76d5d-serviceca\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888365 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-netns\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-netd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-lib-modules\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888419 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-var-lib-kubelet\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888437 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-netd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.956587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-device-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888443 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-run-netns\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-conf-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888483 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-slash\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-slash\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/102018c7-f331-4660-bf47-7d4bc533e188-kube-api-access-k4hqx\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-systemd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-bin\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-system-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888614 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-kubelet\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888628 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-systemd\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-etc-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-host-cni-bin\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-ovn\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-sys\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888666 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-etc-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888688 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-run-ovn\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.957351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-tmp\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888731 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-cnibin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888763 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-daemon-config\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwrqq\" (UniqueName: \"kubernetes.io/projected/3637ef8a-debe-4157-8e92-c74f5c23bb30-kube-api-access-mwrqq\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e27433c-2659-4dc2-9405-b9f3851d935a-agent-certs\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888816 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qv84\" (UniqueName: \"kubernetes.io/projected/47c96d14-5339-496a-8894-e4aeef01e20c-kube-api-access-4qv84\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-k8s-cni-cncf-io\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-netns\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888899 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-env-overrides\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888914 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-systemd-units\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4hbp\" (UniqueName: \"kubernetes.io/projected/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-kube-api-access-v4hbp\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-conf\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87cb950f-3838-4997-a2a4-d85b4291d420-hosts-file\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888968 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.888988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-var-lib-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.958525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jwqp\" (UniqueName: \"kubernetes.io/projected/b7ca6750-e97a-4417-bce0-567a03402717-kube-api-access-9jwqp\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889008 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-systemd-units\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e27433c-2659-4dc2-9405-b9f3851d935a-konnectivity-ca\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-host\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7ca6750-e97a-4417-bce0-567a03402717-var-lib-openvswitch\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.889676 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.891678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-script-lib\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.891723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-ovnkube-config\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.891983 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e27433c-2659-4dc2-9405-b9f3851d935a-konnectivity-ca\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.892043 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7ca6750-e97a-4417-bce0-567a03402717-env-overrides\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.892686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62c48dea-d112-4f60-8d97-5d1be6f76d5d-serviceca\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.892815 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7ca6750-e97a-4417-bce0-567a03402717-ovn-node-metrics-cert\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.893306 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e27433c-2659-4dc2-9405-b9f3851d935a-agent-certs\") pod \"konnectivity-agent-qhggs\" (UID: \"2e27433c-2659-4dc2-9405-b9f3851d935a\") " pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.895196 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmxg7\" (UniqueName: \"kubernetes.io/projected/62c48dea-d112-4f60-8d97-5d1be6f76d5d-kube-api-access-mmxg7\") pod \"node-ca-gncrb\" (UID: \"62c48dea-d112-4f60-8d97-5d1be6f76d5d\") " pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:19.960491 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.895921 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jwqp\" (UniqueName: \"kubernetes.io/projected/b7ca6750-e97a-4417-bce0-567a03402717-kube-api-access-9jwqp\") pod \"ovnkube-node-lmg5b\" (UID: \"b7ca6750-e97a-4417-bce0-567a03402717\") " pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:19.989277 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989215 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-kubelet-config\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.989361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989286 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-kubelet-config\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.989361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989314 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.989361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysconfig\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989345 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-registration-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-modprobe-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q27lc\" (UniqueName: \"kubernetes.io/projected/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-kube-api-access-q27lc\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-sys-fs\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989418 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-registration-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.989439 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-modprobe-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989468 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysconfig\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-kubernetes\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989509 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-sys-fs\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.989515 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:20.489476583 +0000 UTC m=+2.128151339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:19.989525 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.989533 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-kubernetes\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989546 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h24dj\" (UniqueName: \"kubernetes.io/projected/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kube-api-access-h24dj\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.989592 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:20.489574488 +0000 UTC m=+2.128249245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-bin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989702 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-multus\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989726 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989733 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-bin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-cnibin\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989772 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-os-release\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989783 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-cni-multus\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-hostroot\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-multus-certs\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-d\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989836 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-cnibin\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-etc-kubernetes\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989853 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-os-release\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-hostroot\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-dbus\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989887 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-etc-kubernetes\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-iptables-alerter-script\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989898 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-multus-certs\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-host-slash\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.989990 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ae8c8cd-c765-411a-a418-a2387ac28c01-dbus\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-cni-binary-copy\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87cb950f-3838-4997-a2a4-d85b4291d420-tmp-dir\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990060 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-socket-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990083 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-run\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-socket-dir-parent\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990187 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-socket-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.990768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990189 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-lib-modules\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990060 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-host-slash\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-var-lib-kubelet\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-device-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990288 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-run\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-conf-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990328 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-conf-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/102018c7-f331-4660-bf47-7d4bc533e188-kube-api-access-k4hqx\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990370 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-system-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-kubelet\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-sys\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990429 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87cb950f-3838-4997-a2a4-d85b4291d420-tmp-dir\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-tmp\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-cnibin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990507 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-iptables-alerter-script\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990514 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-daemon-config\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.991480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-lib-modules\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990539 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwrqq\" (UniqueName: \"kubernetes.io/projected/3637ef8a-debe-4157-8e92-c74f5c23bb30-kube-api-access-mwrqq\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qv84\" (UniqueName: \"kubernetes.io/projected/47c96d14-5339-496a-8894-e4aeef01e20c-kube-api-access-4qv84\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990581 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-socket-dir-parent\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-k8s-cni-cncf-io\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-netns\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990683 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-system-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4hbp\" (UniqueName: \"kubernetes.io/projected/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-kube-api-access-v4hbp\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-conf\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87cb950f-3838-4997-a2a4-d85b4291d420-hosts-file\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-host\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-tuned\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990836 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-systemd\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.992299 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-systemd\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990982 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-var-lib-kubelet\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991003 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-cni-binary-copy\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991087 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-sysctl-conf\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991090 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-sys\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990593 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-var-lib-kubelet\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991132 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87cb950f-3838-4997-a2a4-d85b4291d420-hosts-file\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-k8s-cni-cncf-io\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990636 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991183 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-host-run-netns\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.990638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-device-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991219 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l874b\" (UniqueName: \"kubernetes.io/projected/87cb950f-3838-4997-a2a4-d85b4291d420-kube-api-access-l874b\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991451 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-system-cni-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991478 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-os-release\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991558 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-os-release\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991581 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/102018c7-f331-4660-bf47-7d4bc533e188-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991599 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-system-cni-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.993003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-host\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991687 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-cnibin\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991757 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-daemon-config\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102018c7-f331-4660-bf47-7d4bc533e188-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.991870 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3637ef8a-debe-4157-8e92-c74f5c23bb30-multus-cni-dir\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.993171 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-tmp\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.993495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:19.993399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-etc-tuned\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:19.999945 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.999924 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:20.000047 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.999949 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:20.000047 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:19.999962 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:20.000047 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.000025 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:20.500009703 +0000 UTC m=+2.138684489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:20.001209 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.001185 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h24dj\" (UniqueName: \"kubernetes.io/projected/913c2e6b-cd1c-4bb8-aede-91d650d9bc76-kube-api-access-h24dj\") pod \"aws-ebs-csi-driver-node-2l4fl\" (UID: \"913c2e6b-cd1c-4bb8-aede-91d650d9bc76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:20.001315 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.001280 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/102018c7-f331-4660-bf47-7d4bc533e188-kube-api-access-k4hqx\") pod \"multus-additional-cni-plugins-zfvtf\" (UID: \"102018c7-f331-4660-bf47-7d4bc533e188\") " pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:20.002296 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.002277 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q27lc\" (UniqueName: \"kubernetes.io/projected/f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c-kube-api-access-q27lc\") pod \"tuned-56knf\" (UID: \"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c\") " pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:20.002438 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.002419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qv84\" (UniqueName: \"kubernetes.io/projected/47c96d14-5339-496a-8894-e4aeef01e20c-kube-api-access-4qv84\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:20.002590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.002571 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4hbp\" (UniqueName: \"kubernetes.io/projected/2d6410a2-ee5f-4739-b908-9d4bfe107ba9-kube-api-access-v4hbp\") pod \"iptables-alerter-h96rp\" (UID: \"2d6410a2-ee5f-4739-b908-9d4bfe107ba9\") " pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:20.002777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.002762 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwrqq\" (UniqueName: \"kubernetes.io/projected/3637ef8a-debe-4157-8e92-c74f5c23bb30-kube-api-access-mwrqq\") pod \"multus-ljtkx\" (UID: \"3637ef8a-debe-4157-8e92-c74f5c23bb30\") " pod="openshift-multus/multus-ljtkx" Jul 09 14:02:20.003435 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.003419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l874b\" (UniqueName: \"kubernetes.io/projected/87cb950f-3838-4997-a2a4-d85b4291d420-kube-api-access-l874b\") pod \"node-resolver-c4r7k\" (UID: \"87cb950f-3838-4997-a2a4-d85b4291d420\") " pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:20.023691 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.023672 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gncrb" Jul 09 14:02:20.035777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.035762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:20.053632 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.053616 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:20.065679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.065666 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" Jul 09 14:02:20.081211 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.081195 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h96rp" Jul 09 14:02:20.104734 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.104719 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" Jul 09 14:02:20.122199 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.122183 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljtkx" Jul 09 14:02:20.137792 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.137770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-56knf" Jul 09 14:02:20.142358 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.142345 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c4r7k" Jul 09 14:02:20.265074 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.265032 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46bed090be62d9ef320eca39a1d183b.slice/crio-ae871368abef1d9b34f1a9d705f5bf4e94f084b17c4cf1240e264f64dbceb499 WatchSource:0}: Error finding container ae871368abef1d9b34f1a9d705f5bf4e94f084b17c4cf1240e264f64dbceb499: Status 404 returned error can't find the container with id ae871368abef1d9b34f1a9d705f5bf4e94f084b17c4cf1240e264f64dbceb499 Jul 09 14:02:20.268743 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.268728 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:02:20.476045 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.476021 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e27433c_2659_4dc2_9405_b9f3851d935a.slice/crio-f4288b34efd21ed3a9b06f3980dbdf191a45a80dbe2891ddbd2b98e5c4c0e6f0 WatchSource:0}: Error finding container f4288b34efd21ed3a9b06f3980dbdf191a45a80dbe2891ddbd2b98e5c4c0e6f0: Status 404 returned error can't find the container with id f4288b34efd21ed3a9b06f3980dbdf191a45a80dbe2891ddbd2b98e5c4c0e6f0 Jul 09 14:02:20.476906 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.476886 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831e3dd915e470f53b8a8ad37f2ef182.slice/crio-71777efd184166ecaf0a57080274986f6ecbac9adff8c384e02b2d87e18358df WatchSource:0}: Error finding container 71777efd184166ecaf0a57080274986f6ecbac9adff8c384e02b2d87e18358df: Status 404 returned error can't find the container with id 71777efd184166ecaf0a57080274986f6ecbac9adff8c384e02b2d87e18358df Jul 09 14:02:20.494569 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.494544 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:20.494660 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.494591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:20.494711 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.494680 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:20.494747 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.494731 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:21.494717222 +0000 UTC m=+3.133391973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:20.494800 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.494685 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:20.494852 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.494826 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:21.494808013 +0000 UTC m=+3.133482776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:20.500340 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.500321 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c48dea_d112_4f60_8d97_5d1be6f76d5d.slice/crio-f1c3edfac0e46b9388482b4256eec0f3b53cd793e1381e24bdb7eb63f9207229 WatchSource:0}: Error finding container f1c3edfac0e46b9388482b4256eec0f3b53cd793e1381e24bdb7eb63f9207229: Status 404 returned error can't find the container with id f1c3edfac0e46b9388482b4256eec0f3b53cd793e1381e24bdb7eb63f9207229 Jul 09 14:02:20.539864 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.539847 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3637ef8a_debe_4157_8e92_c74f5c23bb30.slice/crio-ebddd13ae9771ad778ce314165155a9d0a6b34ec57c2a86dc51fb828d980ddc2 WatchSource:0}: Error finding container ebddd13ae9771ad778ce314165155a9d0a6b34ec57c2a86dc51fb828d980ddc2: Status 404 returned error can't find the container with id ebddd13ae9771ad778ce314165155a9d0a6b34ec57c2a86dc51fb828d980ddc2 Jul 09 14:02:20.556838 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.556818 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e5c1a7_e2ed_4c7a_b5c0_cae4a635885c.slice/crio-e806f836451205c321c3d71a51c169a00fb4693db8f28ed8b1fd580bc9752fdc WatchSource:0}: Error finding container e806f836451205c321c3d71a51c169a00fb4693db8f28ed8b1fd580bc9752fdc: Status 404 returned error can't find the container with id e806f836451205c321c3d71a51c169a00fb4693db8f28ed8b1fd580bc9752fdc Jul 09 14:02:20.595506 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.595485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:20.595602 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.595591 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:20.595648 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.595606 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:20.595648 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.595615 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:20.595723 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:20.595651 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:21.595639148 +0000 UTC m=+3.234313900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:20.751363 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.751341 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:20.754193 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:20.754174 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod102018c7_f331_4660_bf47_7d4bc533e188.slice/crio-0e456131f705ed1643affae408884cf3b9030ae297c4e64b748d976781ce7550 WatchSource:0}: Error finding container 0e456131f705ed1643affae408884cf3b9030ae297c4e64b748d976781ce7550: Status 404 returned error can't find the container with id 0e456131f705ed1643affae408884cf3b9030ae297c4e64b748d976781ce7550 Jul 09 14:02:20.824988 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.824920 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:57:19 +0000 UTC" deadline="2028-04-11 02:49:17.958380052 +0000 UTC" Jul 09 14:02:20.824988 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.824959 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15396h46m57.133423893s" Jul 09 14:02:20.892164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.892113 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljtkx" event={"ID":"3637ef8a-debe-4157-8e92-c74f5c23bb30","Type":"ContainerStarted","Data":"ebddd13ae9771ad778ce314165155a9d0a6b34ec57c2a86dc51fb828d980ddc2"} Jul 09 14:02:20.893057 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.893037 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gncrb" event={"ID":"62c48dea-d112-4f60-8d97-5d1be6f76d5d","Type":"ContainerStarted","Data":"f1c3edfac0e46b9388482b4256eec0f3b53cd793e1381e24bdb7eb63f9207229"} Jul 09 14:02:20.894091 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.894063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"831e3dd915e470f53b8a8ad37f2ef182","Type":"ContainerStarted","Data":"71777efd184166ecaf0a57080274986f6ecbac9adff8c384e02b2d87e18358df"} Jul 09 14:02:20.895194 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.895175 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhggs" event={"ID":"2e27433c-2659-4dc2-9405-b9f3851d935a","Type":"ContainerStarted","Data":"f4288b34efd21ed3a9b06f3980dbdf191a45a80dbe2891ddbd2b98e5c4c0e6f0"} Jul 09 14:02:20.898945 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.898913 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"f46bed090be62d9ef320eca39a1d183b","Type":"ContainerStarted","Data":"ae871368abef1d9b34f1a9d705f5bf4e94f084b17c4cf1240e264f64dbceb499"} Jul 09 14:02:20.900046 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.900012 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerStarted","Data":"0e456131f705ed1643affae408884cf3b9030ae297c4e64b748d976781ce7550"} Jul 09 14:02:20.901119 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:20.901097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-56knf" event={"ID":"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c","Type":"ContainerStarted","Data":"e806f836451205c321c3d71a51c169a00fb4693db8f28ed8b1fd580bc9752fdc"} Jul 09 14:02:21.314329 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:21.314299 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913c2e6b_cd1c_4bb8_aede_91d650d9bc76.slice/crio-398ec25946c758fa97b15a04bbccc18cbda717611bdb9652c4783a12bc051de1 WatchSource:0}: Error finding container 398ec25946c758fa97b15a04bbccc18cbda717611bdb9652c4783a12bc051de1: Status 404 returned error can't find the container with id 398ec25946c758fa97b15a04bbccc18cbda717611bdb9652c4783a12bc051de1 Jul 09 14:02:21.504475 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.504441 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:21.504654 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.504492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:21.504654 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.504606 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:21.504654 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.504621 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:21.504817 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.504675 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:23.504656983 +0000 UTC m=+5.143331740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:21.504817 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.504693 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:23.504684676 +0000 UTC m=+5.143359434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:21.605498 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.605405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:21.605653 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.605585 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:21.605653 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.605611 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:21.605653 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.605625 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:21.605811 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.605686 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:23.605667455 +0000 UTC m=+5.244342220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:21.825850 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.825791 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:57:19 +0000 UTC" deadline="2028-02-01 12:52:46.917677087 +0000 UTC" Jul 09 14:02:21.825850 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.825842 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13726h50m25.09183825s" Jul 09 14:02:21.889328 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.889257 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:21.889328 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.889298 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:21.889328 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.889298 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:21.889580 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.889369 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:21.889580 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.889462 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:21.889580 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:21.889542 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:21.903943 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.903913 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" event={"ID":"913c2e6b-cd1c-4bb8-aede-91d650d9bc76","Type":"ContainerStarted","Data":"398ec25946c758fa97b15a04bbccc18cbda717611bdb9652c4783a12bc051de1"} Jul 09 14:02:21.905055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:21.905032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c4r7k" event={"ID":"87cb950f-3838-4997-a2a4-d85b4291d420","Type":"ContainerStarted","Data":"76370262e3d310714bc96d0bf9ada1d488c09ab3ccadfc8eb041d784866e3a19"} Jul 09 14:02:22.030218 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.030181 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:22.077305 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.077278 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:22.478272 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:22.478246 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ca6750_e97a_4417_bce0_567a03402717.slice/crio-ac99f07bac4e5be57dfba9fc83eb30140ffe72d91a2bb96456e8686ae718a499 WatchSource:0}: Error finding container ac99f07bac4e5be57dfba9fc83eb30140ffe72d91a2bb96456e8686ae718a499: Status 404 returned error can't find the container with id ac99f07bac4e5be57dfba9fc83eb30140ffe72d91a2bb96456e8686ae718a499 Jul 09 14:02:22.554567 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:22.554355 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6410a2_ee5f_4739_b908_9d4bfe107ba9.slice/crio-684824053ce770913eaebcbda351a9352233f60e9544da46749bde8f330634ac WatchSource:0}: Error finding container 684824053ce770913eaebcbda351a9352233f60e9544da46749bde8f330634ac: Status 404 returned error can't find the container with id 684824053ce770913eaebcbda351a9352233f60e9544da46749bde8f330634ac Jul 09 14:02:22.618315 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.618288 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:02:22.909555 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.909518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"ac99f07bac4e5be57dfba9fc83eb30140ffe72d91a2bb96456e8686ae718a499"} Jul 09 14:02:22.912742 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.912703 2576 generic.go:356] "Generic (PLEG): container finished" podID="831e3dd915e470f53b8a8ad37f2ef182" containerID="816fcc55039dab14d45e2bd6a07721e91afaea3ba96572b2a55e4294d4afe20b" exitCode=0 Jul 09 14:02:22.912883 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.912790 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"831e3dd915e470f53b8a8ad37f2ef182","Type":"ContainerDied","Data":"816fcc55039dab14d45e2bd6a07721e91afaea3ba96572b2a55e4294d4afe20b"} Jul 09 14:02:22.916515 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.916479 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"f46bed090be62d9ef320eca39a1d183b","Type":"ContainerStarted","Data":"b6fcafaedb74e734d4eea77c9bb808a182d985526dde55aceb40ac8ed26b6c73"} Jul 09 14:02:22.926561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:22.926467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h96rp" event={"ID":"2d6410a2-ee5f-4739-b908-9d4bfe107ba9","Type":"ContainerStarted","Data":"684824053ce770913eaebcbda351a9352233f60e9544da46749bde8f330634ac"} Jul 09 14:02:23.521017 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.520979 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:23.521199 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.521037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:23.521286 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.521205 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:23.521286 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.521279 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:27.521261188 +0000 UTC m=+9.159935945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:23.521670 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.521652 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:23.521752 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.521706 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:27.521691543 +0000 UTC m=+9.160366299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:23.622386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.622349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:23.622568 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.622533 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:23.622568 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.622559 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:23.622676 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.622572 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:23.622676 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.622630 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:27.622611374 +0000 UTC m=+9.261286131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:23.889174 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.889128 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:23.889353 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.889270 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:23.889648 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.889628 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:23.889735 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.889717 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:23.889791 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.889774 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:23.889874 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:23.889854 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:23.954317 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.954284 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"831e3dd915e470f53b8a8ad37f2ef182","Type":"ContainerStarted","Data":"2278e9add42b61c4a5aaeb579a717fca89f62cb2395f4f986c3b0cde8c8d3185"} Jul 09 14:02:23.972637 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:23.972582 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" podStartSLOduration=4.972561058 podStartE2EDuration="4.972561058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:22.952117853 +0000 UTC m=+4.590792617" watchObservedRunningTime="2026-07-09 14:02:23.972561058 +0000 UTC m=+5.611235832" Jul 09 14:02:25.888755 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:25.888716 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:25.888755 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:25.888744 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:25.889352 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:25.888868 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:25.889352 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:25.888939 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:25.889352 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:25.888993 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:25.889352 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:25.889064 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:27.558472 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.558374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:27.558472 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.558425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:27.558982 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.558563 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:27.558982 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.558643 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:35.558620387 +0000 UTC m=+17.197295141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:27.558982 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.558652 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:27.558982 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.558705 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:35.558688242 +0000 UTC m=+17.197363012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:27.659115 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.659079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:27.659329 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.659250 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:27.659329 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.659269 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:27.659329 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.659280 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:27.659476 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.659341 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:35.659323541 +0000 UTC m=+17.297998308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:27.888725 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.888691 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:27.888918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.888727 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:27.888918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:27.888727 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:27.888918 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.888818 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:27.888918 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.888896 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:27.889124 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:27.888980 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:29.889041 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:29.889005 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:29.889041 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:29.889043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:29.889649 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:29.889136 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:29.889649 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:29.889205 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:29.889649 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:29.889248 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:29.889649 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:29.889345 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:31.889478 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:31.889443 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:31.889925 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:31.889577 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:31.889925 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:31.889590 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:31.889925 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:31.889610 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:31.889925 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:31.889690 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:31.889925 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:31.889774 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:33.888578 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:33.888551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:33.889067 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:33.888551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:33.889067 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:33.888673 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:33.889067 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:33.888551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:33.889067 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:33.888761 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:33.889067 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:33.888812 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:35.625270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.625222 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:35.625684 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.625278 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:35.625684 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.625372 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:35.625684 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.625389 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:35.625684 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.625433 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:02:51.625419685 +0000 UTC m=+33.264094436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:35.625684 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.625447 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:51.62544123 +0000 UTC m=+33.264115982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:35.726434 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.726398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:35.726605 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.726579 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:35.726605 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.726610 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:35.726784 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.726624 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:35.726784 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.726683 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:02:51.726666032 +0000 UTC m=+33.365340800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:35.888448 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.888423 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:35.888574 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.888466 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:35.888574 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:35.888435 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:35.888574 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.888534 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:35.888709 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.888618 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:35.888751 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:35.888701 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:36.983636 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.983401 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="b8498875ad8d9619d6b1d3b29d078d22e99845f0690cd7ce74a69e8bc7edc994" exitCode=0 Jul 09 14:02:36.984273 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.983787 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"b8498875ad8d9619d6b1d3b29d078d22e99845f0690cd7ce74a69e8bc7edc994"} Jul 09 14:02:36.986745 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.986636 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljtkx" event={"ID":"3637ef8a-debe-4157-8e92-c74f5c23bb30","Type":"ContainerStarted","Data":"967d61e8d1221f4277baad761887dfb02bf9e817ff7c7a6e7fb5edc73c88da0f"} Jul 09 14:02:36.988565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.988536 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gncrb" event={"ID":"62c48dea-d112-4f60-8d97-5d1be6f76d5d","Type":"ContainerStarted","Data":"fb27ff13993aa65e14b6852e7a9f366c0003945ecea46880ed7e57c1664771b7"} Jul 09 14:02:36.990178 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.990155 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhggs" event={"ID":"2e27433c-2659-4dc2-9405-b9f3851d935a","Type":"ContainerStarted","Data":"2e6dda0649e6aba0008a1fc694db405be78f36521cdd0df7c7e2d7726c5b1486"} Jul 09 14:02:36.991883 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.991858 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" event={"ID":"913c2e6b-cd1c-4bb8-aede-91d650d9bc76","Type":"ContainerStarted","Data":"00388e605e388110b877d5b30f87d7a331d19882a5abcb36c5bfe821f3c927c2"} Jul 09 14:02:36.993410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.993387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c4r7k" event={"ID":"87cb950f-3838-4997-a2a4-d85b4291d420","Type":"ContainerStarted","Data":"f5047f08d69a57080ebb6c585cbf01ab8001b1f403d0bd4e4b1b5692248d718d"} Jul 09 14:02:36.994906 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:36.994877 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-56knf" event={"ID":"f2e5c1a7-e2ed-4c7a-b5c0-cae4a635885c","Type":"ContainerStarted","Data":"3ce936062a236b0bacf016a5b9099a718066df6f56b454415f6ca7bdec351c6f"} Jul 09 14:02:37.011871 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.011830 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" podStartSLOduration=18.011816605 podStartE2EDuration="18.011816605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:23.973339005 +0000 UTC m=+5.612013780" watchObservedRunningTime="2026-07-09 14:02:37.011816605 +0000 UTC m=+18.650491377" Jul 09 14:02:37.032378 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.032322 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c4r7k" podStartSLOduration=3.44109236 podStartE2EDuration="18.032306823s" totalImagesPullingTime="14.591214463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:37.032257675 +0000 UTC m=+18.670932465" watchObservedRunningTime="2026-07-09 14:02:37.032306823 +0000 UTC m=+18.670981596" Jul 09 14:02:37.073019 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.072977 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qhggs" podStartSLOduration=6.956237787 podStartE2EDuration="18.072963859s" totalImagesPullingTime="11.116726072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:37.072455373 +0000 UTC m=+18.711130166" watchObservedRunningTime="2026-07-09 14:02:37.072963859 +0000 UTC m=+18.711638631" Jul 09 14:02:37.073188 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.073117 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gncrb" podStartSLOduration=3.6944544820000003 podStartE2EDuration="19.073108525s" totalImagesPullingTime="15.378654043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:37.054115359 +0000 UTC m=+18.692790132" watchObservedRunningTime="2026-07-09 14:02:37.073108525 +0000 UTC m=+18.711783299" Jul 09 14:02:37.093752 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.093709 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ljtkx" podStartSLOduration=2.717435648 podStartE2EDuration="18.09369043s" totalImagesPullingTime="15.376254782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:37.093609756 +0000 UTC m=+18.732284531" watchObservedRunningTime="2026-07-09 14:02:37.09369043 +0000 UTC m=+18.732365208" Jul 09 14:02:37.113244 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.113184 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-56knf" podStartSLOduration=2.7897395080000003 podStartE2EDuration="18.113171094s" totalImagesPullingTime="15.323431586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:37.112965026 +0000 UTC m=+18.751639817" watchObservedRunningTime="2026-07-09 14:02:37.113171094 +0000 UTC m=+18.751845918" Jul 09 14:02:37.317750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.317727 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:37.318287 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.318267 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:37.889139 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.889101 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:37.889139 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.889132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:37.889374 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.889132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:37.889374 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:37.889246 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:37.889374 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:37.889315 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:37.889493 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:37.889385 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:37.999012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:37.998888 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h96rp" event={"ID":"2d6410a2-ee5f-4739-b908-9d4bfe107ba9","Type":"ContainerStarted","Data":"a85208630fea5c9dad2f78c30f6289824246d9cc3de3dbd47a17cefaf833f7dc"} Jul 09 14:02:38.020759 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:38.020679 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h96rp" podStartSLOduration=5.695433136 podStartE2EDuration="19.020655134s" totalImagesPullingTime="13.325221998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:38.020534112 +0000 UTC m=+19.659208886" watchObservedRunningTime="2026-07-09 14:02:38.020655134 +0000 UTC m=+19.659329908" Jul 09 14:02:39.000692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:39.000661 2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 14:02:39.888622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:39.888588 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:39.888784 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:39.888588 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:39.888784 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:39.888719 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:39.888784 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:39.888596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:39.888939 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:39.888779 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:39.888939 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:39.888853 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:41.007968 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.007693 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:02:41.008434 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.008412 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7ca6750-e97a-4417-bce0-567a03402717" containerID="d1c5e9890d91c0f95c82a976b930629d53cf6168e2fd7562a999b43eabae85ce" exitCode=1 Jul 09 14:02:41.008515 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.008457 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"97e069b102b69618443b22c311673e2c6da90f89e137037588e94c8f7aa98ee8"} Jul 09 14:02:41.008560 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.008541 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"8fb6c0da65aae671e70dbeb6d7c96d2855449eb2e7983d8de50ebec9bb6bbcb3"} Jul 09 14:02:41.008613 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.008598 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerDied","Data":"d1c5e9890d91c0f95c82a976b930629d53cf6168e2fd7562a999b43eabae85ce"} Jul 09 14:02:41.008665 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.008652 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"1b0228652f594edbf679cd732cb2acf91b65b6d29f7427b6901fbaf430b3706c"} Jul 09 14:02:41.445787 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.445760 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 14:02:41.837773 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.837686 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T14:02:41.445781117Z","UUID":"6f160e17-a0bc-47f7-9650-a32e5f9c919d","Handler":null,"Name":"","Endpoint":""} Jul 09 14:02:41.839001 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.838981 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 14:02:41.839108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.839016 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 14:02:41.888768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.888744 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:41.888905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.888770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:41.888905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:41.888770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:41.888905 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:41.888857 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:41.889044 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:41.888934 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:41.889044 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:41.889010 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:42.013170 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.013135 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" event={"ID":"913c2e6b-cd1c-4bb8-aede-91d650d9bc76","Type":"ContainerStarted","Data":"03a3453a98e0d423415473f474da82f69a745f4546187388289cb7a77979c057"} Jul 09 14:02:42.016279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.016257 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:02:42.016655 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.016634 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"70e070bbe6a1bb5810a033b8855264e330f86073edfdf6aa3e6e53de70b166b2"} Jul 09 14:02:42.016731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.016697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"fd652ad41dd7688d3ae967c4a10963ba2b114292dff0bbc8f2e55f57a1bef378"} Jul 09 14:02:42.087724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.087704 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:42.087879 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.087795 2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 14:02:42.088172 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:42.088147 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qhggs" Jul 09 14:02:43.021038 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:43.020992 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" event={"ID":"913c2e6b-cd1c-4bb8-aede-91d650d9bc76","Type":"ContainerStarted","Data":"5780a6d2e11f17a27369ac986af25b7ff9fa62d3cf76b70b38517dab950d2fff"} Jul 09 14:02:43.041917 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:43.041868 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l4fl" podStartSLOduration=2.866696718 podStartE2EDuration="24.041851392s" totalImagesPullingTime="21.175154674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:43.04153458 +0000 UTC m=+24.680209352" watchObservedRunningTime="2026-07-09 14:02:43.041851392 +0000 UTC m=+24.680526164" Jul 09 14:02:43.889390 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:43.889362 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:43.889554 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:43.889463 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:43.889554 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:43.889466 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:43.889554 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:43.889480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:43.889663 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:43.889552 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:43.889663 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:43.889617 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:44.025607 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:44.025581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:02:44.026080 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:44.025900 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"2958f17bc8ae2ef585592c0c3cdbfa56a61d3e04a12b26d4c8272d4be38664c3"} Jul 09 14:02:45.889004 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:45.888871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:45.889431 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:45.889002 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:45.889431 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:45.889025 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:45.889431 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:45.889147 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:45.889431 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:45.889026 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:45.889431 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:45.889258 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:47.035324 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.035145 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:02:47.035843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.035666 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"1573be86fd7837dde931888835aee361ec535f8eda4f23023e7a33a501a9d97b"} Jul 09 14:02:47.035912 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.035887 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:47.036080 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.036061 2576 scope.go:117] "RemoveContainer" containerID="d1c5e9890d91c0f95c82a976b930629d53cf6168e2fd7562a999b43eabae85ce" Jul 09 14:02:47.037611 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.037590 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="0ce2edde72c773aca5f887b23afebe193c69d52fc3737b86aa1fa422a853c36d" exitCode=0 Jul 09 14:02:47.037722 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.037631 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"0ce2edde72c773aca5f887b23afebe193c69d52fc3737b86aa1fa422a853c36d"} Jul 09 14:02:47.050807 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.050785 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:47.889223 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.889181 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:47.889433 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.889288 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:47.889433 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:47.889293 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:47.889433 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:47.889319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:47.889433 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:47.889394 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:47.889637 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:47.889476 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:48.044051 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.044028 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:02:48.046483 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.046451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" event={"ID":"b7ca6750-e97a-4417-bce0-567a03402717","Type":"ContainerStarted","Data":"408d3fafaf327da3d7bf9c89c765023d33f63e95b6e20ffb10a0273e172f72ab"} Jul 09 14:02:48.046696 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.046676 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:48.046796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.046761 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:48.061509 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.061486 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:02:48.089226 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.089180 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" podStartSLOduration=11.931673049 podStartE2EDuration="30.089166098s" totalImagesPullingTime="18.157493049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:48.087548015 +0000 UTC m=+29.726222787" watchObservedRunningTime="2026-07-09 14:02:48.089166098 +0000 UTC m=+29.727840871" Jul 09 14:02:48.497169 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.497136 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8q4t5"] Jul 09 14:02:48.497354 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.497217 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:48.497354 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:48.497308 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:48.502481 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.502456 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvvs6"] Jul 09 14:02:48.502594 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.502544 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:48.502671 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:48.502653 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:48.517450 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.517422 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dpkkq"] Jul 09 14:02:48.517685 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:48.517517 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:48.517685 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:48.517605 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:49.888856 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:49.888824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:49.889266 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:49.888824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:49.889266 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:49.888927 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:49.889266 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:49.888833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:49.889266 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:49.889040 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:49.889266 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:49.889146 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:51.054015 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.053848 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="e5c479bec9e42a92ca85c865f3ddafba96bf23d002bfbcc78b4966d901c637c3" exitCode=0 Jul 09 14:02:51.054629 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.053925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"e5c479bec9e42a92ca85c865f3ddafba96bf23d002bfbcc78b4966d901c637c3"} Jul 09 14:02:51.648276 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.648243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:51.648422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.648293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:51.648422 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.648406 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:51.648542 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.648475 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.648455999 +0000 UTC m=+65.287130764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:02:51.648542 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.648406 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:51.648649 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.648570 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret podName:7ae8c8cd-c765-411a-a418-a2387ac28c01 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.648547483 +0000 UTC m=+65.287222251 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret") pod "global-pull-secret-syncer-8q4t5" (UID: "7ae8c8cd-c765-411a-a418-a2387ac28c01") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:02:51.749111 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.749083 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:51.749286 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.749265 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:02:51.749344 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.749297 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:02:51.749344 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.749311 2576 projected.go:196] Error preparing data for projected volume kube-api-access-9qrct for pod openshift-network-diagnostics/network-check-target-dpkkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:51.749421 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.749374 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct podName:b25d8bf2-05a9-4301-928d-148ef77d9caa nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.749355804 +0000 UTC m=+65.388030570 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9qrct" (UniqueName: "kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct") pod "network-check-target-dpkkq" (UID: "b25d8bf2-05a9-4301-928d-148ef77d9caa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:02:51.888516 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.888485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:51.888650 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.888485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:51.888650 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.888610 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dpkkq" podUID="b25d8bf2-05a9-4301-928d-148ef77d9caa" Jul 09 14:02:51.888764 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:51.888485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:51.888764 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.888659 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q4t5" podUID="7ae8c8cd-c765-411a-a418-a2387ac28c01" Jul 09 14:02:51.888764 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:51.888730 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvvs6" podUID="47c96d14-5339-496a-8894-e4aeef01e20c" Jul 09 14:02:52.058721 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:52.058655 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="f9fb838c94ec28d2603a63a8f7278c6efc1c3287eb4607494c75f819278797a4" exitCode=0 Jul 09 14:02:52.059094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:52.058722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"f9fb838c94ec28d2603a63a8f7278c6efc1c3287eb4607494c75f819278797a4"} Jul 09 14:02:53.642937 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.642908 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeReady" Jul 09 14:02:53.643506 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.643038 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 14:02:53.678432 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.678399 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:02:53.689066 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.689041 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.692783 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.692739 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 14:02:53.693471 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.693200 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 14:02:53.693841 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.693821 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5cvrb"] Jul 09 14:02:53.693968 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.693946 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zggw5\"" Jul 09 14:02:53.694409 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.694393 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 14:02:53.701416 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.701393 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 14:02:53.713165 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.713140 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:02:53.713308 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.713184 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jczw6"] Jul 09 14:02:53.713308 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.713277 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.715861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.715833 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 14:02:53.715981 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.715897 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s5w7z\"" Jul 09 14:02:53.716075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.716063 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 14:02:53.723050 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.723029 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5cvrb"] Jul 09 14:02:53.723147 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.723061 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jczw6"] Jul 09 14:02:53.723147 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.723125 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.725598 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.725582 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 14:02:53.725715 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.725602 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 14:02:53.725890 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.725802 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 14:02:53.725947 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.725894 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xwx6q\"" Jul 09 14:02:53.866639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866580 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9th9\" (UniqueName: \"kubernetes.io/projected/717b6137-b92d-46b8-9a6f-167d54a214e8-kube-api-access-n9th9\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.866639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.866639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qcb\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.866847 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.866847 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.866847 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866730 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/717b6137-b92d-46b8-9a6f-167d54a214e8-config-volume\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.866847 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866756 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-cert\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.866847 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866849 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8svbt\" (UniqueName: \"kubernetes.io/projected/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-kube-api-access-8svbt\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.866921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.867011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b6137-b92d-46b8-9a6f-167d54a214e8-tmp-dir\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.867072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.867045 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.888835 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.888811 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:02:53.888959 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.888847 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:02:53.888959 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.888811 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:02:53.892220 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892142 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:02:53.892220 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892191 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27nh2\"" Jul 09 14:02:53.892408 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892202 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:02:53.892408 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892203 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:02:53.892408 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892151 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:02:53.892408 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.892217 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wp6bz\"" Jul 09 14:02:53.968103 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.968278 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968278 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968278 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968183 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8svbt\" (UniqueName: \"kubernetes.io/projected/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-kube-api-access-8svbt\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.968278 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968205 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968278 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:53.968257 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b6137-b92d-46b8-9a6f-167d54a214e8-tmp-dir\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:53.968310 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:54.468292763 +0000 UTC m=+36.106967515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968377 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9th9\" (UniqueName: \"kubernetes.io/projected/717b6137-b92d-46b8-9a6f-167d54a214e8-kube-api-access-n9th9\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qcb\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/717b6137-b92d-46b8-9a6f-167d54a214e8-config-volume\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.968987 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-cert\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.968987 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968569 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.968987 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.968575 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b6137-b92d-46b8-9a6f-167d54a214e8-tmp-dir\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.968987 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:53.968673 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:02:53.968987 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:53.968686 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:02:53.969205 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:53.969017 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:54.468988586 +0000 UTC m=+36.107663351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:02:53.969205 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.969028 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.969205 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.969087 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/717b6137-b92d-46b8-9a6f-167d54a214e8-config-volume\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:53.969932 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.969900 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.973360 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.973180 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.973483 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.973191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.973483 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.973218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-cert\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.977948 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.977867 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.978092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.978067 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qcb\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:53.978198 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.978181 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8svbt\" (UniqueName: \"kubernetes.io/projected/c302bd4d-03dc-48a3-aa09-0e14dfc592d0-kube-api-access-8svbt\") pod \"ingress-canary-jczw6\" (UID: \"c302bd4d-03dc-48a3-aa09-0e14dfc592d0\") " pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:53.978623 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:53.978604 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9th9\" (UniqueName: \"kubernetes.io/projected/717b6137-b92d-46b8-9a6f-167d54a214e8-kube-api-access-n9th9\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:54.031058 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.031036 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jczw6" Jul 09 14:02:54.228272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.228242 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jczw6"] Jul 09 14:02:54.233330 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:02:54.233298 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc302bd4d_03dc_48a3_aa09_0e14dfc592d0.slice/crio-e1a237eda2f769a75d29a766f7c95a3ac90e0d9c81aea853dc5976495c8dfa33 WatchSource:0}: Error finding container e1a237eda2f769a75d29a766f7c95a3ac90e0d9c81aea853dc5976495c8dfa33: Status 404 returned error can't find the container with id e1a237eda2f769a75d29a766f7c95a3ac90e0d9c81aea853dc5976495c8dfa33 Jul 09 14:02:54.411349 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.411279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:02:54.473334 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.473304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:54.473484 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.473391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:54.473484 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:54.473471 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:02:54.473566 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:54.473533 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:55.47351897 +0000 UTC m=+37.112193721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:02:54.473616 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:54.473477 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:02:54.473616 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:54.473577 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:02:54.473721 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:54.473628 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:55.47361186 +0000 UTC m=+37.112286626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:02:54.499324 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:54.499288 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:02:55.066988 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:55.066951 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jczw6" event={"ID":"c302bd4d-03dc-48a3-aa09-0e14dfc592d0","Type":"ContainerStarted","Data":"e1a237eda2f769a75d29a766f7c95a3ac90e0d9c81aea853dc5976495c8dfa33"} Jul 09 14:02:55.483375 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:55.483310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:55.483375 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:55.483373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:55.483578 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:55.483471 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:02:55.483578 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:55.483511 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:57.483500029 +0000 UTC m=+39.122174780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:02:55.483578 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:55.483559 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:02:55.483719 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:55.483580 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:02:55.483719 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:55.483646 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:02:57.483628245 +0000 UTC m=+39.122303011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:02:57.500390 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:57.500354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:57.500526 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:57.500541 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:57.500549 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:57.500609 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:57.500682 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:01.500662726 +0000 UTC m=+43.139337478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:02:57.500773 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:02:57.500700 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:01.500688878 +0000 UTC m=+43.139363638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:02:59.078083 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:59.078049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jczw6" event={"ID":"c302bd4d-03dc-48a3-aa09-0e14dfc592d0","Type":"ContainerStarted","Data":"66599403e8a424477554340107ddd4e690cf0c180e4232577e0521b9e15d2f29"} Jul 09 14:02:59.080731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:59.080704 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="702ce4ca65740acc8e9a34d06822cc1a046dd3347f012233a27781febe812314" exitCode=0 Jul 09 14:02:59.080843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:59.080742 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"702ce4ca65740acc8e9a34d06822cc1a046dd3347f012233a27781febe812314"} Jul 09 14:02:59.095211 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:02:59.095169 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jczw6" podStartSLOduration=1.925489749 podStartE2EDuration="6.095140419s" totalImagesPullingTime="4.16965067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:59.094767537 +0000 UTC m=+40.733442329" watchObservedRunningTime="2026-07-09 14:02:59.095140419 +0000 UTC m=+40.733815186" Jul 09 14:03:00.086049 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:00.086013 2576 generic.go:356] "Generic (PLEG): container finished" podID="102018c7-f331-4660-bf47-7d4bc533e188" containerID="068694034e4c32299dfd06da92de1c749e22c74b032f841a33443a9ac6ef0cf2" exitCode=0 Jul 09 14:03:00.086394 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:00.086095 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerDied","Data":"068694034e4c32299dfd06da92de1c749e22c74b032f841a33443a9ac6ef0cf2"} Jul 09 14:03:00.092630 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:00.092600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:03:00.195907 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:00.195847 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:03:01.091558 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.091520 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" event={"ID":"102018c7-f331-4660-bf47-7d4bc533e188","Type":"ContainerStarted","Data":"21923be2639bf6e5f50cff032b39f91b4cea27054899f833bf2a116df7186a05"} Jul 09 14:03:01.117463 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.117403 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zfvtf" podStartSLOduration=7.421342568 podStartE2EDuration="42.117382972s" totalImagesPullingTime="34.696040404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:01.115483443 +0000 UTC m=+42.754158216" watchObservedRunningTime="2026-07-09 14:03:01.117382972 +0000 UTC m=+42.756057747" Jul 09 14:03:01.239143 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.239108 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:03:01.339919 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.339888 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:03:01.532437 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.532374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:01.532437 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:01.532427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:01.532584 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:01.532509 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:03:01.532584 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:01.532567 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:09.532553874 +0000 UTC m=+51.171228625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:03:01.532661 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:01.532511 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:03:01.532661 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:01.532601 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:03:01.532661 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:01.532647 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:09.532636539 +0000 UTC m=+51.171311290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:03:03.961899 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:03.961866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" limit=1024 Jul 09 14:03:09.590573 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:09.590540 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:09.590607 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:09.590694 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:09.590708 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:09.590719 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:09.590757 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.59074121 +0000 UTC m=+67.229415961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:03:09.591034 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:09.590773 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.590764028 +0000 UTC m=+67.229438779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:03:20.069980 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:20.069948 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lmg5b" Jul 09 14:03:21.781674 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.781642 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr"] Jul 09 14:03:21.784601 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.784581 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" Jul 09 14:03:21.785833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.785810 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz"] Jul 09 14:03:21.787812 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.787790 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w"] Jul 09 14:03:21.787921 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.787907 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" Jul 09 14:03:21.789347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.789327 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl"] Jul 09 14:03:21.789469 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.789450 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.790928 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.790898 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.790928 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.790920 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gsp45\"" Jul 09 14:03:21.791095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.790990 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.791095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.791088 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-jzqf9"] Jul 09 14:03:21.791196 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.791153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.791820 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.791802 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rzvw7\"" Jul 09 14:03:21.792671 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792621 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.792671 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792638 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 14:03:21.792671 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792640 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.792851 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792739 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 14:03:21.792851 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792763 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.792979 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.792952 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m2n59\"" Jul 09 14:03:21.793622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.793609 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6kjj2\"" Jul 09 14:03:21.795556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795536 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.795556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795550 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.795705 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795660 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 14:03:21.795795 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795780 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 14:03:21.795887 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795870 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.795946 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.795911 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.796139 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.796027 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jqgn2\"" Jul 09 14:03:21.796271 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.796251 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 14:03:21.796271 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.796261 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 14:03:21.797071 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.797040 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr"] Jul 09 14:03:21.799825 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.799808 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz"] Jul 09 14:03:21.800788 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.800760 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w"] Jul 09 14:03:21.801788 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.801766 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl"] Jul 09 14:03:21.803392 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.803370 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 14:03:21.804814 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.804794 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jzqf9"] Jul 09 14:03:21.881600 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzf68\" (UniqueName: \"kubernetes.io/projected/a2e67427-f275-404c-b9bc-e73e99c38d97-kube-api-access-pzf68\") pod \"volume-data-source-validator-78495597f6-dspmr\" (UID: \"a2e67427-f275-404c-b9bc-e73e99c38d97\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" Jul 09 14:03:21.881600 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qbmw\" (UniqueName: \"kubernetes.io/projected/84261dd3-3238-4f0b-b2bc-c453c9963162-kube-api-access-5qbmw\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.881801 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a47e50e-4829-4881-bea1-bbeebe26be21-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.881801 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881654 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a47e50e-4829-4881-bea1-bbeebe26be21-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.881801 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881670 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/205237ba-20da-4c38-8008-9a4403f17b1f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.881801 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vhn\" (UniqueName: \"kubernetes.io/projected/205237ba-20da-4c38-8008-9a4403f17b1f-kube-api-access-w8vhn\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.881801 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-config\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.881958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881816 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvr6m\" (UniqueName: \"kubernetes.io/projected/ad7180f4-addf-420d-aa8f-d67e5432bbbb-kube-api-access-bvr6m\") pod \"network-check-source-869dcfcdcd-kjqkz\" (UID: \"ad7180f4-addf-420d-aa8f-d67e5432bbbb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" Jul 09 14:03:21.881958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84261dd3-3238-4f0b-b2bc-c453c9963162-serving-cert\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.881958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2w5w\" (UniqueName: \"kubernetes.io/projected/7a47e50e-4829-4881-bea1-bbeebe26be21-kube-api-access-n2w5w\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.881958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881878 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-trusted-ca\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.881958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.881898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.884026 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.884001 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh"] Jul 09 14:03:21.886845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.886820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:21.891174 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891139 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p9xm2"] Jul 09 14:03:21.891323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891203 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.891323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891276 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 14:03:21.891323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891279 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xc47d\"" Jul 09 14:03:21.891323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891302 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 14:03:21.891645 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.891629 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.892903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.892887 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67bb56f78d-q42xs"] Jul 09 14:03:21.893055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.893036 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.894772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.894756 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9"] Jul 09 14:03:21.894862 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.894850 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.896444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896425 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 14:03:21.896548 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896520 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.896609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896521 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:21.896609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896567 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ddv6f\"" Jul 09 14:03:21.896709 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896627 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.896821 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.896803 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 14:03:21.897377 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.897362 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4q2zs\"" Jul 09 14:03:21.897554 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.897542 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.897986 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.897972 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.898062 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.897976 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 14:03:21.898892 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.898874 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 14:03:21.898892 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.898905 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 14:03:21.898892 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.898929 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 14:03:21.899502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.899472 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 14:03:21.899602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.899506 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:21.899602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.899517 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:21.899693 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.899605 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-c7ffq\"" Jul 09 14:03:21.901526 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.901508 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh"] Jul 09 14:03:21.903670 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.903654 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 14:03:21.908559 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.908513 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p9xm2"] Jul 09 14:03:21.909350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.909330 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9"] Jul 09 14:03:21.911634 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.911612 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67bb56f78d-q42xs"] Jul 09 14:03:21.982249 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a47e50e-4829-4881-bea1-bbeebe26be21-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.982417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg2zl\" (UniqueName: \"kubernetes.io/projected/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-kube-api-access-zg2zl\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.982417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982299 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-config\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.982417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.982417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84261dd3-3238-4f0b-b2bc-c453c9963162-serving-cert\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.982417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2w5w\" (UniqueName: \"kubernetes.io/projected/7a47e50e-4829-4881-bea1-bbeebe26be21-kube-api-access-n2w5w\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:21.982424 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7mc\" (UniqueName: \"kubernetes.io/projected/0dece533-f971-48c8-972c-c312fe875473-kube-api-access-jv7mc\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-trusted-ca\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982495 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:21.982507 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:22.48248913 +0000 UTC m=+64.121163883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982585 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-stats-auth\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982614 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-snapshots\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.982682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982694 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a47e50e-4829-4881-bea1-bbeebe26be21-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/205237ba-20da-4c38-8008-9a4403f17b1f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982780 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vhn\" (UniqueName: \"kubernetes.io/projected/205237ba-20da-4c38-8008-9a4403f17b1f-kube-api-access-w8vhn\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982805 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4nr\" (UniqueName: \"kubernetes.io/projected/050a74e3-9444-46b7-863e-17c6b5148bad-kube-api-access-7p4nr\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.982942 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a47e50e-4829-4881-bea1-bbeebe26be21-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983004 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbmw\" (UniqueName: \"kubernetes.io/projected/84261dd3-3238-4f0b-b2bc-c453c9963162-kube-api-access-5qbmw\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983087 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983114 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-config\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvr6m\" (UniqueName: \"kubernetes.io/projected/ad7180f4-addf-420d-aa8f-d67e5432bbbb-kube-api-access-bvr6m\") pod \"network-check-source-869dcfcdcd-kjqkz\" (UID: \"ad7180f4-addf-420d-aa8f-d67e5432bbbb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" Jul 09 14:03:21.983164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/050a74e3-9444-46b7-863e-17c6b5148bad-config\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-serving-cert\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983210 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/050a74e3-9444-46b7-863e-17c6b5148bad-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wwct\" (UniqueName: \"kubernetes.io/projected/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-kube-api-access-4wwct\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-default-certificate\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzf68\" (UniqueName: \"kubernetes.io/projected/a2e67427-f275-404c-b9bc-e73e99c38d97-kube-api-access-pzf68\") pod \"volume-data-source-validator-78495597f6-dspmr\" (UID: \"a2e67427-f275-404c-b9bc-e73e99c38d97\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-tmp\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983565 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84261dd3-3238-4f0b-b2bc-c453c9963162-trusted-ca\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.983588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.983573 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/205237ba-20da-4c38-8008-9a4403f17b1f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:21.986604 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.986575 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a47e50e-4829-4881-bea1-bbeebe26be21-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.986762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.986657 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84261dd3-3238-4f0b-b2bc-c453c9963162-serving-cert\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:21.987954 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.987935 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pw5d6"] Jul 09 14:03:21.995558 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.995540 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:21.998166 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.998143 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jp8fc\"" Jul 09 14:03:21.998308 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.998293 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 14:03:21.998420 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.998403 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 14:03:21.999583 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.999564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2w5w\" (UniqueName: \"kubernetes.io/projected/7a47e50e-4829-4881-bea1-bbeebe26be21-kube-api-access-n2w5w\") pod \"kube-storage-version-migrator-operator-799cb7f958-m5j9w\" (UID: \"7a47e50e-4829-4881-bea1-bbeebe26be21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:21.999762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:21.999740 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vhn\" (UniqueName: \"kubernetes.io/projected/205237ba-20da-4c38-8008-9a4403f17b1f-kube-api-access-w8vhn\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:22.000495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.000473 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pw5d6"] Jul 09 14:03:22.002343 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.002318 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzf68\" (UniqueName: \"kubernetes.io/projected/a2e67427-f275-404c-b9bc-e73e99c38d97-kube-api-access-pzf68\") pod \"volume-data-source-validator-78495597f6-dspmr\" (UID: \"a2e67427-f275-404c-b9bc-e73e99c38d97\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" Jul 09 14:03:22.002763 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.002735 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:03:22.002945 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.002923 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qbmw\" (UniqueName: \"kubernetes.io/projected/84261dd3-3238-4f0b-b2bc-c453c9963162-kube-api-access-5qbmw\") pod \"console-operator-7866497494-jzqf9\" (UID: \"84261dd3-3238-4f0b-b2bc-c453c9963162\") " pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:22.013478 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.013458 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:03:22.023868 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.023845 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvr6m\" (UniqueName: \"kubernetes.io/projected/ad7180f4-addf-420d-aa8f-d67e5432bbbb-kube-api-access-bvr6m\") pod \"network-check-source-869dcfcdcd-kjqkz\" (UID: \"ad7180f4-addf-420d-aa8f-d67e5432bbbb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" Jul 09 14:03:22.083920 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.083895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/050a74e3-9444-46b7-863e-17c6b5148bad-config\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.084090 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.083929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-serving-cert\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084090 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.083952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.084090 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.083970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/050a74e3-9444-46b7-863e-17c6b5148bad-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.084090 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wwct\" (UniqueName: \"kubernetes.io/projected/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-kube-api-access-4wwct\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:22.084347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084094 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-default-certificate\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.084347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-tmp\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg2zl\" (UniqueName: \"kubernetes.io/projected/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-kube-api-access-zg2zl\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7mc\" (UniqueName: \"kubernetes.io/projected/0dece533-f971-48c8-972c-c312fe875473-kube-api-access-jv7mc\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.084495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.084495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084422 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:22.084495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084463 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-stats-auth\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.084495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084486 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-snapshots\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084540 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-tmp\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.084596 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:22.584574499 +0000 UTC m=+64.223249256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4nr\" (UniqueName: \"kubernetes.io/projected/050a74e3-9444-46b7-863e-17c6b5148bad-kube-api-access-7p4nr\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.084724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.085007 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.084749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b62edca-81e3-4de9-88e0-bd08cd290874-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.085007 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.084842 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 14:03:22.085007 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.084912 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:22.085346 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.084917 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls podName:f16c14f3-4d7f-4da2-9194-134e1dd80a6d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:22.584884208 +0000 UTC m=+64.223558963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls") pod "cluster-samples-operator-65db858894-h94d9" (UID: "f16c14f3-4d7f-4da2-9194-134e1dd80a6d") : secret "samples-operator-tls" not found Jul 09 14:03:22.085346 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.085100 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:22.585080194 +0000 UTC m=+64.223754954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:22.085346 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.085141 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.085660 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.085485 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-snapshots\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.085660 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.085640 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/050a74e3-9444-46b7-863e-17c6b5148bad-config\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.085844 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.085816 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.086502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.086481 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/050a74e3-9444-46b7-863e-17c6b5148bad-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.087142 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.087122 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-serving-cert\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.087268 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.087254 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-default-certificate\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.087316 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.087255 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-stats-auth\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.093148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.093129 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg2zl\" (UniqueName: \"kubernetes.io/projected/9243e7ec-aa19-4e2c-adeb-18debfd84b6f-kube-api-access-zg2zl\") pod \"insights-operator-68d9dd94cf-p9xm2\" (UID: \"9243e7ec-aa19-4e2c-adeb-18debfd84b6f\") " pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.093422 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.093403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wwct\" (UniqueName: \"kubernetes.io/projected/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-kube-api-access-4wwct\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:22.093604 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.093588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7mc\" (UniqueName: \"kubernetes.io/projected/0dece533-f971-48c8-972c-c312fe875473-kube-api-access-jv7mc\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.093803 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.093788 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4nr\" (UniqueName: \"kubernetes.io/projected/050a74e3-9444-46b7-863e-17c6b5148bad-kube-api-access-7p4nr\") pod \"service-ca-operator-7695fc6c8c-mxxwh\" (UID: \"050a74e3-9444-46b7-863e-17c6b5148bad\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.095763 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.095740 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" Jul 09 14:03:22.104931 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.104916 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" Jul 09 14:03:22.111020 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.110999 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" Jul 09 14:03:22.122493 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.122471 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:22.185585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.185556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b62edca-81e3-4de9-88e0-bd08cd290874-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.185718 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.185606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.186372 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.185848 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:22.186372 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.185919 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:22.685899882 +0000 UTC m=+64.324574637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:22.186965 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.186940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b62edca-81e3-4de9-88e0-bd08cd290874-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.197129 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.196875 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" Jul 09 14:03:22.206082 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.206050 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" Jul 09 14:03:22.251653 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.248493 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr"] Jul 09 14:03:22.256599 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:22.256553 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e67427_f275_404c_b9bc_e73e99c38d97.slice/crio-e78df8bbb53a2f11337e71205b7b16dfd6ba49081a9fab605dd4b5dc8157586c WatchSource:0}: Error finding container e78df8bbb53a2f11337e71205b7b16dfd6ba49081a9fab605dd4b5dc8157586c: Status 404 returned error can't find the container with id e78df8bbb53a2f11337e71205b7b16dfd6ba49081a9fab605dd4b5dc8157586c Jul 09 14:03:22.276134 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.276029 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz"] Jul 09 14:03:22.303256 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.303107 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w"] Jul 09 14:03:22.304531 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.304505 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jzqf9"] Jul 09 14:03:22.305279 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:22.305254 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a47e50e_4829_4881_bea1_bbeebe26be21.slice/crio-01d05862f28be31cf4bbcdd5f7ccdea6c939accc2907d49fb1a5a09db387a38d WatchSource:0}: Error finding container 01d05862f28be31cf4bbcdd5f7ccdea6c939accc2907d49fb1a5a09db387a38d: Status 404 returned error can't find the container with id 01d05862f28be31cf4bbcdd5f7ccdea6c939accc2907d49fb1a5a09db387a38d Jul 09 14:03:22.307770 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:22.307644 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84261dd3_3238_4f0b_b2bc_c453c9963162.slice/crio-8fab71f43d047cc193be6ed58c412c133f2f873f800ad2acefb1c61f7fb7f9d1 WatchSource:0}: Error finding container 8fab71f43d047cc193be6ed58c412c133f2f873f800ad2acefb1c61f7fb7f9d1: Status 404 returned error can't find the container with id 8fab71f43d047cc193be6ed58c412c133f2f873f800ad2acefb1c61f7fb7f9d1 Jul 09 14:03:22.367802 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.367724 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh"] Jul 09 14:03:22.370826 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:22.370801 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050a74e3_9444_46b7_863e_17c6b5148bad.slice/crio-eda861400bd3dc40eab1e56f97b28f661de799dd4cc35843ba08c38ac900dec0 WatchSource:0}: Error finding container eda861400bd3dc40eab1e56f97b28f661de799dd4cc35843ba08c38ac900dec0: Status 404 returned error can't find the container with id eda861400bd3dc40eab1e56f97b28f661de799dd4cc35843ba08c38ac900dec0 Jul 09 14:03:22.381647 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.381624 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p9xm2"] Jul 09 14:03:22.384577 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:22.384554 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9243e7ec_aa19_4e2c_adeb_18debfd84b6f.slice/crio-746765e5755fd0ec7c1e77d426b3c6d39c6b8a1c588a69adedba02fe65cb205c WatchSource:0}: Error finding container 746765e5755fd0ec7c1e77d426b3c6d39c6b8a1c588a69adedba02fe65cb205c: Status 404 returned error can't find the container with id 746765e5755fd0ec7c1e77d426b3c6d39c6b8a1c588a69adedba02fe65cb205c Jul 09 14:03:22.489348 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.489306 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:22.489517 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.489467 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:22.489563 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.489529 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.489512231 +0000 UTC m=+65.128187004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:22.589913 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.589877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.589913 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.589917 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:22.590127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.589993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:22.590127 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.590049 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.590030952 +0000 UTC m=+65.228705709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:22.590127 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.590103 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 14:03:22.590274 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.590148 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls podName:f16c14f3-4d7f-4da2-9194-134e1dd80a6d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.590136952 +0000 UTC m=+65.228811708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls") pod "cluster-samples-operator-65db858894-h94d9" (UID: "f16c14f3-4d7f-4da2-9194-134e1dd80a6d") : secret "samples-operator-tls" not found Jul 09 14:03:22.590274 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.590103 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:22.590274 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.590177 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.590171601 +0000 UTC m=+65.228846358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:22.690803 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:22.690705 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:22.690968 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.690840 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:22.690968 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:22.690906 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:23.690888033 +0000 UTC m=+65.329562787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:23.144121 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.144021 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerStarted","Data":"8fab71f43d047cc193be6ed58c412c133f2f873f800ad2acefb1c61f7fb7f9d1"} Jul 09 14:03:23.145493 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.145379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" event={"ID":"9243e7ec-aa19-4e2c-adeb-18debfd84b6f","Type":"ContainerStarted","Data":"746765e5755fd0ec7c1e77d426b3c6d39c6b8a1c588a69adedba02fe65cb205c"} Jul 09 14:03:23.147347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.147310 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" event={"ID":"ad7180f4-addf-420d-aa8f-d67e5432bbbb","Type":"ContainerStarted","Data":"14984004df9220306ab0f4ce30b300f0df14f06d871246ffbdd380919a7b990e"} Jul 09 14:03:23.148938 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.148907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" event={"ID":"a2e67427-f275-404c-b9bc-e73e99c38d97","Type":"ContainerStarted","Data":"e78df8bbb53a2f11337e71205b7b16dfd6ba49081a9fab605dd4b5dc8157586c"} Jul 09 14:03:23.150649 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.150604 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" event={"ID":"050a74e3-9444-46b7-863e-17c6b5148bad","Type":"ContainerStarted","Data":"eda861400bd3dc40eab1e56f97b28f661de799dd4cc35843ba08c38ac900dec0"} Jul 09 14:03:23.151983 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.151958 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" event={"ID":"7a47e50e-4829-4881-bea1-bbeebe26be21","Type":"ContainerStarted","Data":"01d05862f28be31cf4bbcdd5f7ccdea6c939accc2907d49fb1a5a09db387a38d"} Jul 09 14:03:23.500065 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.499379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:23.500065 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.499609 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:23.500065 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.499670 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.499650967 +0000 UTC m=+67.138325726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:23.600354 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.600312 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:23.600558 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.600366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:23.600558 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.600509 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.600482374 +0000 UTC m=+67.239157146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:23.600558 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.600520 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 14:03:23.600726 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.600575 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls podName:f16c14f3-4d7f-4da2-9194-134e1dd80a6d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.600559519 +0000 UTC m=+67.239234273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls") pod "cluster-samples-operator-65db858894-h94d9" (UID: "f16c14f3-4d7f-4da2-9194-134e1dd80a6d") : secret "samples-operator-tls" not found Jul 09 14:03:23.600726 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.600685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:23.600840 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.600787 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:23.600840 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.600824 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.60081266 +0000 UTC m=+67.239487417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:23.701519 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.701478 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:03:23.701706 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.701535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:03:23.701706 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.701593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:23.701825 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.701760 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:23.701825 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.701821 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:25.701803896 +0000 UTC m=+67.340478652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:23.704692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.704659 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:03:23.704692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.704689 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:03:23.712927 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.712794 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 14:03:23.712927 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:23.712876 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs podName:47c96d14-5339-496a-8894-e4aeef01e20c nodeName:}" failed. No retries permitted until 2026-07-09 14:04:27.712853363 +0000 UTC m=+129.351528122 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs") pod "network-metrics-daemon-tvvs6" (UID: "47c96d14-5339-496a-8894-e4aeef01e20c") : secret "metrics-daemon-secret" not found Jul 09 14:03:23.717137 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.717060 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ae8c8cd-c765-411a-a418-a2387ac28c01-original-pull-secret\") pod \"global-pull-secret-syncer-8q4t5\" (UID: \"7ae8c8cd-c765-411a-a418-a2387ac28c01\") " pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:03:23.803171 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.803023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:03:23.814297 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.814217 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qrct\" (UniqueName: \"kubernetes.io/projected/b25d8bf2-05a9-4301-928d-148ef77d9caa-kube-api-access-9qrct\") pod \"network-check-target-dpkkq\" (UID: \"b25d8bf2-05a9-4301-928d-148ef77d9caa\") " pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:03:23.899829 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.899792 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q4t5" Jul 09 14:03:23.916084 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.916053 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wp6bz\"" Jul 09 14:03:23.923396 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:23.923367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:03:24.157735 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:24.157696 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dpkkq"] Jul 09 14:03:24.174991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:24.174866 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8q4t5"] Jul 09 14:03:24.770845 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:24.770808 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25d8bf2_05a9_4301_928d_148ef77d9caa.slice/crio-bd2c1eb3f9015dc95fa298467503c126e20a60281539b432e1403ffb759d46e2 WatchSource:0}: Error finding container bd2c1eb3f9015dc95fa298467503c126e20a60281539b432e1403ffb759d46e2: Status 404 returned error can't find the container with id bd2c1eb3f9015dc95fa298467503c126e20a60281539b432e1403ffb759d46e2 Jul 09 14:03:25.160168 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.160125 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" event={"ID":"a2e67427-f275-404c-b9bc-e73e99c38d97","Type":"ContainerStarted","Data":"994db02e16d903d2ba1daf1b43a0450aefb4a76431e539fd4b5a6b5a4e2daefd"} Jul 09 14:03:25.161490 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.161453 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dpkkq" event={"ID":"b25d8bf2-05a9-4301-928d-148ef77d9caa","Type":"ContainerStarted","Data":"bd2c1eb3f9015dc95fa298467503c126e20a60281539b432e1403ffb759d46e2"} Jul 09 14:03:25.176246 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.176168 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dspmr" podStartSLOduration=2.402157609 podStartE2EDuration="4.176147695s" totalImagesPullingTime="1.773990086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:25.175795195 +0000 UTC m=+66.814469970" watchObservedRunningTime="2026-07-09 14:03:25.176147695 +0000 UTC m=+66.814822471" Jul 09 14:03:25.302436 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:25.302401 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae8c8cd_c765_411a_a418_a2387ac28c01.slice/crio-8631a25bcca87db414dba6624765ef12866fa31a7af2bd8e7589ab3b32ca3d40 WatchSource:0}: Error finding container 8631a25bcca87db414dba6624765ef12866fa31a7af2bd8e7589ab3b32ca3d40: Status 404 returned error can't find the container with id 8631a25bcca87db414dba6624765ef12866fa31a7af2bd8e7589ab3b32ca3d40 Jul 09 14:03:25.520641 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.520553 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:25.520799 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.520734 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:25.520870 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.520815 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:29.52079475 +0000 UTC m=+71.159469516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:25.621501 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.621448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:25.621692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.621501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:25.621692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.621554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:25.621692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.621590 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:25.621692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.621607 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:25.621900 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.621749 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:25.621900 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.621798 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:29.621785163 +0000 UTC m=+71.260459915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:25.622124 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622097 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:03:25.622192 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622130 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bbf4cb7c-v58tr: secret "image-registry-tls" not found Jul 09 14:03:25.622192 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622138 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 14:03:25.622192 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622103 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:03:25.622373 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622133 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:29.622089211 +0000 UTC m=+71.260763978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:25.622373 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622247 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls podName:edad103a-e449-4533-a278-54aa63dd07d6 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:57.622213999 +0000 UTC m=+99.260888762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls") pod "image-registry-86bbf4cb7c-v58tr" (UID: "edad103a-e449-4533-a278-54aa63dd07d6") : secret "image-registry-tls" not found Jul 09 14:03:25.622373 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622263 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls podName:f16c14f3-4d7f-4da2-9194-134e1dd80a6d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:29.622254206 +0000 UTC m=+71.260928962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls") pod "cluster-samples-operator-65db858894-h94d9" (UID: "f16c14f3-4d7f-4da2-9194-134e1dd80a6d") : secret "samples-operator-tls" not found Jul 09 14:03:25.622373 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.622287 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls podName:717b6137-b92d-46b8-9a6f-167d54a214e8 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:57.622279246 +0000 UTC m=+99.260953999 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls") pod "dns-default-5cvrb" (UID: "717b6137-b92d-46b8-9a6f-167d54a214e8") : secret "dns-default-metrics-tls" not found Jul 09 14:03:25.722647 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:25.722596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:25.722827 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.722798 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:25.722889 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:25.722877 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:29.722854897 +0000 UTC m=+71.361529668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:26.165163 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:26.165122 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8q4t5" event={"ID":"7ae8c8cd-c765-411a-a418-a2387ac28c01","Type":"ContainerStarted","Data":"8631a25bcca87db414dba6624765ef12866fa31a7af2bd8e7589ab3b32ca3d40"} Jul 09 14:03:27.170874 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.170805 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerStarted","Data":"b9cdfa8550c735f1413e5534c4a380864db3e7de3477a8639d1713f60f5cc1f8"} Jul 09 14:03:27.171260 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.170941 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:27.172698 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.172667 2576 patch_prober.go:28] interesting pod/console-operator-7866497494-jzqf9 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body= Jul 09 14:03:27.172829 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.172723 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" Jul 09 14:03:27.173463 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.173402 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" event={"ID":"ad7180f4-addf-420d-aa8f-d67e5432bbbb","Type":"ContainerStarted","Data":"be6ea6c39c1fa72015f4d6b39744572b1eaf2777777756585ef6be9bde4a2e0f"} Jul 09 14:03:27.175219 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.175119 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" event={"ID":"7a47e50e-4829-4881-bea1-bbeebe26be21","Type":"ContainerStarted","Data":"b8b252b6435c1b2738607e3e2834433752dfed3d074c0521faf12b4ad51b9435"} Jul 09 14:03:27.187941 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.187894 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podStartSLOduration=1.529335341 podStartE2EDuration="6.187877896s" totalImagesPullingTime="4.658542555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:27.187060921 +0000 UTC m=+68.825735696" watchObservedRunningTime="2026-07-09 14:03:27.187877896 +0000 UTC m=+68.826552670" Jul 09 14:03:27.202524 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.202435 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kjqkz" podStartSLOduration=1.503245052 podStartE2EDuration="6.202421213s" totalImagesPullingTime="4.699176161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:27.201908654 +0000 UTC m=+68.840583429" watchObservedRunningTime="2026-07-09 14:03:27.202421213 +0000 UTC m=+68.841095986" Jul 09 14:03:27.216776 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:27.216732 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" podStartSLOduration=1.5584498249999998 podStartE2EDuration="6.216718465s" totalImagesPullingTime="4.65826864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:27.216268899 +0000 UTC m=+68.854943673" watchObservedRunningTime="2026-07-09 14:03:27.216718465 +0000 UTC m=+68.855393275" Jul 09 14:03:28.180162 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.180136 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/0.log" Jul 09 14:03:28.180546 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.180176 2576 generic.go:356] "Generic (PLEG): container finished" podID="84261dd3-3238-4f0b-b2bc-c453c9963162" containerID="b9cdfa8550c735f1413e5534c4a380864db3e7de3477a8639d1713f60f5cc1f8" exitCode=255 Jul 09 14:03:28.180546 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.180281 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerDied","Data":"b9cdfa8550c735f1413e5534c4a380864db3e7de3477a8639d1713f60f5cc1f8"} Jul 09 14:03:28.180546 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.180499 2576 scope.go:117] "RemoveContainer" containerID="b9cdfa8550c735f1413e5534c4a380864db3e7de3477a8639d1713f60f5cc1f8" Jul 09 14:03:28.181964 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.181914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dpkkq" event={"ID":"b25d8bf2-05a9-4301-928d-148ef77d9caa","Type":"ContainerStarted","Data":"ca6943fafe534694c941b2db8017a5ab85d1d3c66b0507b2d70888bb3c925e23"} Jul 09 14:03:28.183582 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.183559 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" event={"ID":"050a74e3-9444-46b7-863e-17c6b5148bad","Type":"ContainerStarted","Data":"a3de11b68b56cc8ecb068f87423e1c657dafb92ed7b680adc4ec822d609daedc"} Jul 09 14:03:28.219869 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:28.219820 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" podStartSLOduration=2.154179217 podStartE2EDuration="7.219802041s" totalImagesPullingTime="5.065622824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:28.219153118 +0000 UTC m=+69.857827893" watchObservedRunningTime="2026-07-09 14:03:28.219802041 +0000 UTC m=+69.858476815" Jul 09 14:03:29.189873 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.189839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/1.log" Jul 09 14:03:29.190492 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.190473 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/0.log" Jul 09 14:03:29.190587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.190518 2576 generic.go:356] "Generic (PLEG): container finished" podID="84261dd3-3238-4f0b-b2bc-c453c9963162" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" exitCode=255 Jul 09 14:03:29.190642 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.190605 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerDied","Data":"c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804"} Jul 09 14:03:29.190684 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.190671 2576 scope.go:117] "RemoveContainer" containerID="b9cdfa8550c735f1413e5534c4a380864db3e7de3477a8639d1713f60f5cc1f8" Jul 09 14:03:29.191048 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.191004 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:29.191668 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.191189 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:29.193740 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.193100 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" event={"ID":"9243e7ec-aa19-4e2c-adeb-18debfd84b6f","Type":"ContainerStarted","Data":"22ec3235fffcd269c4496a837b852767d7179d8328fa5ef1cd9c225486f4dabc"} Jul 09 14:03:29.193740 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.193145 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:03:29.223633 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.223434 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" podStartSLOduration=2.11984049 podStartE2EDuration="8.223417449s" totalImagesPullingTime="6.103576959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:29.222731625 +0000 UTC m=+70.861406422" watchObservedRunningTime="2026-07-09 14:03:29.223417449 +0000 UTC m=+70.862092224" Jul 09 14:03:29.239265 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.237496 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dpkkq" podStartSLOduration=67.573081975 podStartE2EDuration="1m10.237474854s" totalImagesPullingTime="2.664392879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:29.23719552 +0000 UTC m=+70.875870292" watchObservedRunningTime="2026-07-09 14:03:29.237474854 +0000 UTC m=+70.876149610" Jul 09 14:03:29.561650 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.561608 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:29.561818 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.561773 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:29.561876 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.561852 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:37.561831134 +0000 UTC m=+79.200505887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:29.663321 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.663282 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:29.663321 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.663329 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.663404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.663510 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:37.663482324 +0000 UTC m=+79.302157089 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.663526 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.663576 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:37.663559963 +0000 UTC m=+79.302234737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.663527 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 14:03:29.663636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.663616 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls podName:f16c14f3-4d7f-4da2-9194-134e1dd80a6d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:37.663606507 +0000 UTC m=+79.302281266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls") pod "cluster-samples-operator-65db858894-h94d9" (UID: "f16c14f3-4d7f-4da2-9194-134e1dd80a6d") : secret "samples-operator-tls" not found Jul 09 14:03:29.764348 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:29.764308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:29.764535 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.764479 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:29.764605 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:29.764560 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:37.764538921 +0000 UTC m=+79.403213677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:30.198670 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.198636 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/1.log" Jul 09 14:03:30.199054 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.199020 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:30.199249 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:30.199212 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:30.619037 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.618993 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gl4lx"] Jul 09 14:03:30.624572 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.624546 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.627763 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.627731 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 14:03:30.628006 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.627986 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 14:03:30.628295 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.628275 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zvwkr\"" Jul 09 14:03:30.635302 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.635277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gl4lx"] Jul 09 14:03:30.774008 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.773970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4accd7b0-8f25-4292-9483-34a75cf0cc9d-data-volume\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.774196 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.774069 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.774196 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.774122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.774359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.774209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znlmj\" (UniqueName: \"kubernetes.io/projected/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-api-access-znlmj\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.774359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.774310 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4accd7b0-8f25-4292-9483-34a75cf0cc9d-crio-socket\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875196 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875113 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znlmj\" (UniqueName: \"kubernetes.io/projected/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-api-access-znlmj\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875379 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:30.875326 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:30.875379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4accd7b0-8f25-4292-9483-34a75cf0cc9d-crio-socket\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875577 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:30.875386 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:31.375367686 +0000 UTC m=+73.014042438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:30.875577 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4accd7b0-8f25-4292-9483-34a75cf0cc9d-data-volume\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875577 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875436 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4accd7b0-8f25-4292-9483-34a75cf0cc9d-crio-socket\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.875577 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:30.875477 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:03:30.875577 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:30.875546 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:31.375526921 +0000 UTC m=+73.014201697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : configmap "kube-rbac-proxy" not found Jul 09 14:03:30.875855 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.875835 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4accd7b0-8f25-4292-9483-34a75cf0cc9d-data-volume\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:30.884316 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:30.884287 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znlmj\" (UniqueName: \"kubernetes.io/projected/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-api-access-znlmj\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:31.382392 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:31.382348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:31.382849 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:31.382435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:31.382849 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:31.382374 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:03:31.382849 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:31.382518 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:32.382495359 +0000 UTC m=+74.021170124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : configmap "kube-rbac-proxy" not found Jul 09 14:03:31.382849 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:31.382575 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:31.382849 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:31.382629 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:32.382613744 +0000 UTC m=+74.021288600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:32.123561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:32.123522 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:32.123958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:32.123934 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:32.124147 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:32.124122 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:32.392032 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:32.391937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:32.392032 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:32.392030 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:32.392636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:32.392081 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:03:32.392636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:32.392184 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:34.392148067 +0000 UTC m=+76.030822823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : configmap "kube-rbac-proxy" not found Jul 09 14:03:32.392636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:32.392215 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:32.392636 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:32.392329 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:34.392293735 +0000 UTC m=+76.030968490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:33.210298 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:33.210259 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8q4t5" event={"ID":"7ae8c8cd-c765-411a-a418-a2387ac28c01","Type":"ContainerStarted","Data":"e9748d4fd5bf10d2743095c98d2495254ec05231b78fafcc38798a101ace2f8a"} Jul 09 14:03:33.215663 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:33.215640 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c4r7k_87cb950f-3838-4997-a2a4-d85b4291d420/dns-node-resolver/0.log" Jul 09 14:03:33.230065 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:33.230017 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8q4t5" podStartSLOduration=67.076557835 podStartE2EDuration="1m14.229972628s" totalImagesPullingTime="7.153414793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:33.229175653 +0000 UTC m=+74.867850426" watchObservedRunningTime="2026-07-09 14:03:33.229972628 +0000 UTC m=+74.868647401" Jul 09 14:03:34.408132 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:34.408095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:34.408602 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:34.408265 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:34.408602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:34.408281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:34.408602 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:34.408324 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:38.408308809 +0000 UTC m=+80.046983562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:34.408602 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:34.408368 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:03:34.408602 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:34.408426 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:38.408407913 +0000 UTC m=+80.047082670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : configmap "kube-rbac-proxy" not found Jul 09 14:03:34.418408 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:34.418385 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gncrb_62c48dea-d112-4f60-8d97-5d1be6f76d5d/node-ca/0.log" Jul 09 14:03:35.416005 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:35.415974 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jczw6_c302bd4d-03dc-48a3-aa09-0e14dfc592d0/serve-healthcheck-canary/0.log" Jul 09 14:03:36.017546 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:36.017518 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-m5j9w_7a47e50e-4829-4881-bea1-bbeebe26be21/kube-storage-version-migrator-operator/0.log" Jul 09 14:03:37.171127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.171087 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:37.171525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.171488 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:37.171658 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.171637 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:37.634152 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.634101 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:37.634366 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.634270 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:37.634366 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.634350 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls podName:205237ba-20da-4c38-8008-9a4403f17b1f nodeName:}" failed. No retries permitted until 2026-07-09 14:03:53.63433309 +0000 UTC m=+95.273007847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-nwltl" (UID: "205237ba-20da-4c38-8008-9a4403f17b1f") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:03:37.735197 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.735152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:37.735197 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.735201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:37.735471 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.735311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:37.735471 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.735349 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:53.735330962 +0000 UTC m=+95.374005717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : configmap references non-existent config key: service-ca.crt Jul 09 14:03:37.735471 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.735401 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:03:37.735471 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.735467 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs podName:0dece533-f971-48c8-972c-c312fe875473 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:53.735451486 +0000 UTC m=+95.374126252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs") pod "router-default-67bb56f78d-q42xs" (UID: "0dece533-f971-48c8-972c-c312fe875473") : secret "router-metrics-certs-default" not found Jul 09 14:03:37.737623 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.737593 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f16c14f3-4d7f-4da2-9194-134e1dd80a6d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-h94d9\" (UID: \"f16c14f3-4d7f-4da2-9194-134e1dd80a6d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:37.820370 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.820324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" Jul 09 14:03:37.836475 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.836450 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:37.836637 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.836616 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:03:37.836718 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:37.836695 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert podName:2b62edca-81e3-4de9-88e0-bd08cd290874 nodeName:}" failed. No retries permitted until 2026-07-09 14:03:53.836674321 +0000 UTC m=+95.475349075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pw5d6" (UID: "2b62edca-81e3-4de9-88e0-bd08cd290874") : secret "networking-console-plugin-cert" not found Jul 09 14:03:37.967406 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:37.967371 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9"] Jul 09 14:03:38.228205 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:38.228106 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" event={"ID":"f16c14f3-4d7f-4da2-9194-134e1dd80a6d","Type":"ContainerStarted","Data":"d6fa6a84ecfbf614c90818f3402637b8fb79a4e9b1ec2a416defe31fdb7a8ec8"} Jul 09 14:03:38.441462 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:38.441418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:38.441641 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:38.441492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:38.441641 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:38.441543 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:03:38.441641 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:38.441594 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:38.441641 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:38.441600 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:46.441583914 +0000 UTC m=+88.080258673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : configmap "kube-rbac-proxy" not found Jul 09 14:03:38.441802 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:38.441701 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:03:46.44169029 +0000 UTC m=+88.080365042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:41.240473 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:41.240438 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" event={"ID":"f16c14f3-4d7f-4da2-9194-134e1dd80a6d","Type":"ContainerStarted","Data":"d7b52afd5769bb39d9a22aa53d26ae37f26e4d1fa0801f918039332c1f28c242"} Jul 09 14:03:41.240849 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:41.240488 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" event={"ID":"f16c14f3-4d7f-4da2-9194-134e1dd80a6d","Type":"ContainerStarted","Data":"d7efa36b79a9f2cd628376b189541c754a68ed2934ebbd01f537034101016e66"} Jul 09 14:03:46.512346 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:46.512304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:46.512745 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:46.512358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:46.512745 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:46.512469 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:03:46.512745 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:46.512521 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls podName:4accd7b0-8f25-4292-9483-34a75cf0cc9d nodeName:}" failed. No retries permitted until 2026-07-09 14:04:02.512506507 +0000 UTC m=+104.151181258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gl4lx" (UID: "4accd7b0-8f25-4292-9483-34a75cf0cc9d") : secret "insights-runtime-extractor-tls" not found Jul 09 14:03:46.512909 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:46.512885 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4accd7b0-8f25-4292-9483-34a75cf0cc9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:03:51.889200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:51.889165 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:52.273175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273099 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:03:52.273520 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273505 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/1.log" Jul 09 14:03:52.273566 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273540 2576 generic.go:356] "Generic (PLEG): container finished" podID="84261dd3-3238-4f0b-b2bc-c453c9963162" containerID="d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e" exitCode=255 Jul 09 14:03:52.273600 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273574 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerDied","Data":"d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e"} Jul 09 14:03:52.273632 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273618 2576 scope.go:117] "RemoveContainer" containerID="c420ede1399939fca72772bf6a76c46109b9bfd9440fa7964829e176bc1a2804" Jul 09 14:03:52.273967 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.273946 2576 scope.go:117] "RemoveContainer" containerID="d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e" Jul 09 14:03:52.274175 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:52.274140 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:52.292865 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:52.292818 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-h94d9" podStartSLOduration=28.81304615 podStartE2EDuration="31.292803465s" totalImagesPullingTime="2.479757315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:41.257179399 +0000 UTC m=+82.895854172" watchObservedRunningTime="2026-07-09 14:03:52.292803465 +0000 UTC m=+93.931478276" Jul 09 14:03:53.277706 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.277679 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:03:53.670356 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.670318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:53.672788 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.672763 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/205237ba-20da-4c38-8008-9a4403f17b1f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-nwltl\" (UID: \"205237ba-20da-4c38-8008-9a4403f17b1f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:53.771248 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.771200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:53.771440 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.771317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:53.771850 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.771821 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dece533-f971-48c8-972c-c312fe875473-service-ca-bundle\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:53.773579 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.773559 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dece533-f971-48c8-972c-c312fe875473-metrics-certs\") pod \"router-default-67bb56f78d-q42xs\" (UID: \"0dece533-f971-48c8-972c-c312fe875473\") " pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:53.872536 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.872500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:53.874746 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.874726 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b62edca-81e3-4de9-88e0-bd08cd290874-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pw5d6\" (UID: \"2b62edca-81e3-4de9-88e0-bd08cd290874\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:53.917650 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:53.917621 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" Jul 09 14:03:54.014750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.014721 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:54.043031 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.043007 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl"] Jul 09 14:03:54.045149 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:54.045105 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205237ba_20da_4c38_8008_9a4403f17b1f.slice/crio-812b74e358d8146a6421556e56605922e638fc9d144a3503c789dac81cb48466 WatchSource:0}: Error finding container 812b74e358d8146a6421556e56605922e638fc9d144a3503c789dac81cb48466: Status 404 returned error can't find the container with id 812b74e358d8146a6421556e56605922e638fc9d144a3503c789dac81cb48466 Jul 09 14:03:54.115797 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.115769 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" Jul 09 14:03:54.143042 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.143017 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67bb56f78d-q42xs"] Jul 09 14:03:54.145439 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:54.145410 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dece533_f971_48c8_972c_c312fe875473.slice/crio-6562f2d8909232d181c9ab35975077415b29bb9ad9c15bd99c563f25e1edb5ec WatchSource:0}: Error finding container 6562f2d8909232d181c9ab35975077415b29bb9ad9c15bd99c563f25e1edb5ec: Status 404 returned error can't find the container with id 6562f2d8909232d181c9ab35975077415b29bb9ad9c15bd99c563f25e1edb5ec Jul 09 14:03:54.244386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.244350 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pw5d6"] Jul 09 14:03:54.247486 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:54.247460 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b62edca_81e3_4de9_88e0_bd08cd290874.slice/crio-67b679b2bd17b425c2a01d7800787256c5787a66061bcb403a6ecf4f3e3d8923 WatchSource:0}: Error finding container 67b679b2bd17b425c2a01d7800787256c5787a66061bcb403a6ecf4f3e3d8923: Status 404 returned error can't find the container with id 67b679b2bd17b425c2a01d7800787256c5787a66061bcb403a6ecf4f3e3d8923 Jul 09 14:03:54.282942 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.282912 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67bb56f78d-q42xs" event={"ID":"0dece533-f971-48c8-972c-c312fe875473","Type":"ContainerStarted","Data":"6b704459121d0921713a668137414f4e5d723948ec172481a849a945be7bffcd"} Jul 09 14:03:54.283359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.282966 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67bb56f78d-q42xs" event={"ID":"0dece533-f971-48c8-972c-c312fe875473","Type":"ContainerStarted","Data":"6562f2d8909232d181c9ab35975077415b29bb9ad9c15bd99c563f25e1edb5ec"} Jul 09 14:03:54.284123 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.284093 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" event={"ID":"2b62edca-81e3-4de9-88e0-bd08cd290874","Type":"ContainerStarted","Data":"67b679b2bd17b425c2a01d7800787256c5787a66061bcb403a6ecf4f3e3d8923"} Jul 09 14:03:54.285112 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.285091 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" event={"ID":"205237ba-20da-4c38-8008-9a4403f17b1f","Type":"ContainerStarted","Data":"812b74e358d8146a6421556e56605922e638fc9d144a3503c789dac81cb48466"} Jul 09 14:03:54.306369 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:54.306323 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67bb56f78d-q42xs" podStartSLOduration=33.306308502 podStartE2EDuration="33.306308502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:54.305417654 +0000 UTC m=+95.944092428" watchObservedRunningTime="2026-07-09 14:03:54.306308502 +0000 UTC m=+95.944983253" Jul 09 14:03:55.015311 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:55.015277 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:55.018087 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:55.018062 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:55.289555 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:55.289454 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:55.291044 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:55.291020 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67bb56f78d-q42xs" Jul 09 14:03:57.171274 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.171207 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:03:57.171658 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.171633 2576 scope.go:117] "RemoveContainer" containerID="d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e" Jul 09 14:03:57.171810 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:03:57.171790 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:03:57.303187 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.303151 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" event={"ID":"2b62edca-81e3-4de9-88e0-bd08cd290874","Type":"ContainerStarted","Data":"b5b0b705e29c77a24a8b74512fa58d00a42d0b22f2a13fde06617ded188fe346"} Jul 09 14:03:57.304544 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.304518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" event={"ID":"205237ba-20da-4c38-8008-9a4403f17b1f","Type":"ContainerStarted","Data":"cc7f3e4022e6ed1076d795fdf22c29510671898db005493d2307673b24113724"} Jul 09 14:03:57.320247 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.320121 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-pw5d6" podStartSLOduration=34.091467369 podStartE2EDuration="36.320106797s" totalImagesPullingTime="2.228639428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:57.319760126 +0000 UTC m=+98.958434899" watchObservedRunningTime="2026-07-09 14:03:57.320106797 +0000 UTC m=+98.958781570" Jul 09 14:03:57.341473 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.341422 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-nwltl" podStartSLOduration=33.911867791 podStartE2EDuration="36.341404992s" totalImagesPullingTime="2.429537201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:57.340622614 +0000 UTC m=+98.979297407" watchObservedRunningTime="2026-07-09 14:03:57.341404992 +0000 UTC m=+98.980079764" Jul 09 14:03:57.705249 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.705184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:57.705420 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.705322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:57.707724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.707692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/717b6137-b92d-46b8-9a6f-167d54a214e8-metrics-tls\") pod \"dns-default-5cvrb\" (UID: \"717b6137-b92d-46b8-9a6f-167d54a214e8\") " pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:57.707816 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.707746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"image-registry-86bbf4cb7c-v58tr\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:57.904537 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.904506 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zggw5\"" Jul 09 14:03:57.912718 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.912690 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:57.924436 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.924409 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s5w7z\"" Jul 09 14:03:57.932543 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:57.932522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5cvrb" Jul 09 14:03:58.079175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.079151 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:03:58.080161 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:58.080135 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedad103a_e449_4533_a278_54aa63dd07d6.slice/crio-35ffdc90e050a09484daafa8066b2d17c23423735586eb0d679c10d514a24fb3 WatchSource:0}: Error finding container 35ffdc90e050a09484daafa8066b2d17c23423735586eb0d679c10d514a24fb3: Status 404 returned error can't find the container with id 35ffdc90e050a09484daafa8066b2d17c23423735586eb0d679c10d514a24fb3 Jul 09 14:03:58.090510 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.090489 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5cvrb"] Jul 09 14:03:58.092193 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:03:58.092167 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717b6137_b92d_46b8_9a6f_167d54a214e8.slice/crio-0ebc15ff1e07e27ee2e74554c1e4655683ab45177a5ef2f2755bd2a48f3284f6 WatchSource:0}: Error finding container 0ebc15ff1e07e27ee2e74554c1e4655683ab45177a5ef2f2755bd2a48f3284f6: Status 404 returned error can't find the container with id 0ebc15ff1e07e27ee2e74554c1e4655683ab45177a5ef2f2755bd2a48f3284f6 Jul 09 14:03:58.309659 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.309625 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" event={"ID":"edad103a-e449-4533-a278-54aa63dd07d6","Type":"ContainerStarted","Data":"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7"} Jul 09 14:03:58.310042 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.309676 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" event={"ID":"edad103a-e449-4533-a278-54aa63dd07d6","Type":"ContainerStarted","Data":"35ffdc90e050a09484daafa8066b2d17c23423735586eb0d679c10d514a24fb3"} Jul 09 14:03:58.310042 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.309733 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:03:58.310798 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.310773 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cvrb" event={"ID":"717b6137-b92d-46b8-9a6f-167d54a214e8","Type":"ContainerStarted","Data":"0ebc15ff1e07e27ee2e74554c1e4655683ab45177a5ef2f2755bd2a48f3284f6"} Jul 09 14:03:58.330744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:03:58.330695 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" podStartSLOduration=99.330681846 podStartE2EDuration="1m39.330681846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:58.32922184 +0000 UTC m=+99.967896634" watchObservedRunningTime="2026-07-09 14:03:58.330681846 +0000 UTC m=+99.969356617" Jul 09 14:04:00.202609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:00.202577 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dpkkq" Jul 09 14:04:00.324035 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:00.323956 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cvrb" event={"ID":"717b6137-b92d-46b8-9a6f-167d54a214e8","Type":"ContainerStarted","Data":"9a084e0a6f0d2c742bf9f22a812b8ac798c3b96e124f1b3e3abc790b6479a377"} Jul 09 14:04:00.324035 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:00.324031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cvrb" event={"ID":"717b6137-b92d-46b8-9a6f-167d54a214e8","Type":"ContainerStarted","Data":"2b99cc347dbf43e0bcae4e154f5a590f620dc334908113bdc8bd3c38ecde0744"} Jul 09 14:04:01.327575 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:01.327539 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5cvrb" Jul 09 14:04:01.345692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:01.345635 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5cvrb" podStartSLOduration=66.371388179 podStartE2EDuration="1m8.345615314s" totalImagesPullingTime="1.974227135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:01.345044824 +0000 UTC m=+102.983719630" watchObservedRunningTime="2026-07-09 14:04:01.345615314 +0000 UTC m=+102.984290087" Jul 09 14:04:02.123580 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.123546 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:04:02.124008 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.123978 2576 scope.go:117] "RemoveContainer" containerID="d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e" Jul 09 14:04:02.124171 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:02.124152 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-jzqf9_openshift-console-operator(84261dd3-3238-4f0b-b2bc-c453c9963162)\"" pod="openshift-console-operator/console-operator-7866497494-jzqf9" podUID="84261dd3-3238-4f0b-b2bc-c453c9963162" Jul 09 14:04:02.538641 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.538530 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:04:02.540865 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.540839 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4accd7b0-8f25-4292-9483-34a75cf0cc9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gl4lx\" (UID: \"4accd7b0-8f25-4292-9483-34a75cf0cc9d\") " pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:04:02.737188 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.737149 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gl4lx" Jul 09 14:04:02.862027 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:02.861998 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gl4lx"] Jul 09 14:04:02.864856 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:02.864830 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4accd7b0_8f25_4292_9483_34a75cf0cc9d.slice/crio-3be80de49fe23ac362f206ff76b90a8fe1d33783d39e76f3caa5ed3c1d696e37 WatchSource:0}: Error finding container 3be80de49fe23ac362f206ff76b90a8fe1d33783d39e76f3caa5ed3c1d696e37: Status 404 returned error can't find the container with id 3be80de49fe23ac362f206ff76b90a8fe1d33783d39e76f3caa5ed3c1d696e37 Jul 09 14:04:03.336712 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:03.336674 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gl4lx" event={"ID":"4accd7b0-8f25-4292-9483-34a75cf0cc9d","Type":"ContainerStarted","Data":"3be80de49fe23ac362f206ff76b90a8fe1d33783d39e76f3caa5ed3c1d696e37"} Jul 09 14:04:07.355506 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:07.355470 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gl4lx" event={"ID":"4accd7b0-8f25-4292-9483-34a75cf0cc9d","Type":"ContainerStarted","Data":"7f32aca1d9ad47de289adddc14648f74081d646e6ba3d08bd983ebb5cca2ed30"} Jul 09 14:04:08.360314 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:08.360265 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gl4lx" event={"ID":"4accd7b0-8f25-4292-9483-34a75cf0cc9d","Type":"ContainerStarted","Data":"ab54f428ebd17642e131782255a0ef4b487698960275400686536883f815d976"} Jul 09 14:04:11.332713 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:11.332682 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5cvrb" Jul 09 14:04:12.505377 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.505333 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9sc26"] Jul 09 14:04:12.509334 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.509313 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x6zvx"] Jul 09 14:04:12.509450 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.509431 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.512403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.512380 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 14:04:12.512523 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.512417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 14:04:12.512588 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.512572 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.513389 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.513365 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bw7jc\"" Jul 09 14:04:12.513389 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.513378 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 14:04:12.513592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.513367 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 14:04:12.515273 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.515256 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 14:04:12.515428 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.515412 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t8b2p\"" Jul 09 14:04:12.515570 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.515532 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 14:04:12.515722 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.515706 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 14:04:12.522145 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.522112 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9sc26"] Jul 09 14:04:12.625036 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625002 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625036 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t2l8\" (UniqueName: \"kubernetes.io/projected/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-api-access-7t2l8\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mscdr\" (UniqueName: \"kubernetes.io/projected/a17465a6-b481-4031-aa11-42699ff72664-kube-api-access-mscdr\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0deb68b5-ebd8-42f1-81b8-a9422077801d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625117 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-textfile\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-wtmp\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625146 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625271 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-accelerators-collector-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.625565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-metrics-client-ca\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-sys\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.625565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.625389 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-root\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726486 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.726562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726533 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.726562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-accelerators-collector-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-metrics-client-ca\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-sys\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-root\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-sys\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t2l8\" (UniqueName: \"kubernetes.io/projected/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-api-access-7t2l8\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mscdr\" (UniqueName: \"kubernetes.io/projected/a17465a6-b481-4031-aa11-42699ff72664-kube-api-access-mscdr\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.726843 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726875 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0deb68b5-ebd8-42f1-81b8-a9422077801d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726902 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-textfile\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.726925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-wtmp\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727048 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-root\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727100 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-wtmp\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:12.727179 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:12.727285 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls podName:a17465a6-b481-4031-aa11-42699ff72664 nodeName:}" failed. No retries permitted until 2026-07-09 14:04:13.227263104 +0000 UTC m=+114.865937858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls") pod "node-exporter-x6zvx" (UID: "a17465a6-b481-4031-aa11-42699ff72664") : secret "node-exporter-tls" not found Jul 09 14:04:12.727350 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727347 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-metrics-client-ca\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727423 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.727710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0deb68b5-ebd8-42f1-81b8-a9422077801d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.727710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727570 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-textfile\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.727821 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727761 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.727975 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.727955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-accelerators-collector-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.729592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.729566 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.729816 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.729799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.729874 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.729848 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.739980 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.739952 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t2l8\" (UniqueName: \"kubernetes.io/projected/0deb68b5-ebd8-42f1-81b8-a9422077801d-kube-api-access-7t2l8\") pod \"kube-state-metrics-785778ff74-9sc26\" (UID: \"0deb68b5-ebd8-42f1-81b8-a9422077801d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.740174 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.740152 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mscdr\" (UniqueName: \"kubernetes.io/projected/a17465a6-b481-4031-aa11-42699ff72664-kube-api-access-mscdr\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:12.821981 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.821945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" Jul 09 14:04:12.895222 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.895171 2576 scope.go:117] "RemoveContainer" containerID="d76359049c7bef3ba47f5fa1b4502c4892af2891a1fc7b92c355584b0366816e" Jul 09 14:04:12.966175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:12.966144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9sc26"] Jul 09 14:04:12.970272 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:12.970244 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0deb68b5_ebd8_42f1_81b8_a9422077801d.slice/crio-aa5c469eac9e7e1dcb293611b4385765b9843a2e22f7ffe52f10a2b703becdb0 WatchSource:0}: Error finding container aa5c469eac9e7e1dcb293611b4385765b9843a2e22f7ffe52f10a2b703becdb0: Status 404 returned error can't find the container with id aa5c469eac9e7e1dcb293611b4385765b9843a2e22f7ffe52f10a2b703becdb0 Jul 09 14:04:13.231049 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.231015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:13.233259 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.233242 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a17465a6-b481-4031-aa11-42699ff72664-node-exporter-tls\") pod \"node-exporter-x6zvx\" (UID: \"a17465a6-b481-4031-aa11-42699ff72664\") " pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:13.382994 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.382961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" event={"ID":"0deb68b5-ebd8-42f1-81b8-a9422077801d","Type":"ContainerStarted","Data":"aa5c469eac9e7e1dcb293611b4385765b9843a2e22f7ffe52f10a2b703becdb0"} Jul 09 14:04:13.384663 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.384643 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:04:13.384774 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.384728 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jzqf9" event={"ID":"84261dd3-3238-4f0b-b2bc-c453c9963162","Type":"ContainerStarted","Data":"5705afc41bdba7ce67121c0d50283df396228c70e8520914dc4d078006c106fc"} Jul 09 14:04:13.385028 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.384996 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:04:13.387059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.387039 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gl4lx" event={"ID":"4accd7b0-8f25-4292-9483-34a75cf0cc9d","Type":"ContainerStarted","Data":"95afc676d650acbd95f778f9136879d0b10c4f66df3ab6cd8f4b22d5aa3cff49"} Jul 09 14:04:13.427875 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.427845 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x6zvx" Jul 09 14:04:13.439448 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.439404 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gl4lx" podStartSLOduration=34.040581785 podStartE2EDuration="43.439387666s" totalImagesPullingTime="9.398805881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:03:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:13.438467702 +0000 UTC m=+115.077142493" watchObservedRunningTime="2026-07-09 14:04:13.439387666 +0000 UTC m=+115.078062438" Jul 09 14:04:13.439599 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:13.439575 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17465a6_b481_4031_aa11_42699ff72664.slice/crio-0871a84491b6b712346a1761979cce10fe02a41c92f5f58b4fd13214d6d382ad WatchSource:0}: Error finding container 0871a84491b6b712346a1761979cce10fe02a41c92f5f58b4fd13214d6d382ad: Status 404 returned error can't find the container with id 0871a84491b6b712346a1761979cce10fe02a41c92f5f58b4fd13214d6d382ad Jul 09 14:04:13.482132 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:13.482046 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-jzqf9" Jul 09 14:04:14.393138 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.393060 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x6zvx" event={"ID":"a17465a6-b481-4031-aa11-42699ff72664","Type":"ContainerStarted","Data":"0871a84491b6b712346a1761979cce10fe02a41c92f5f58b4fd13214d6d382ad"} Jul 09 14:04:14.550563 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.550525 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-69b765cb6-4nsxc"] Jul 09 14:04:14.557146 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.557108 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.559965 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.559938 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 14:04:14.560111 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.560064 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6vuk6hun687n7\"" Jul 09 14:04:14.560175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.560120 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k8tt2\"" Jul 09 14:04:14.560439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.560281 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 14:04:14.560439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.560297 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 14:04:14.560439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.560311 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 14:04:14.561031 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.561009 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 14:04:14.566215 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.566193 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69b765cb6-4nsxc"] Jul 09 14:04:14.646039 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.645950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-grpc-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646039 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646263 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646263 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646108 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4v4\" (UniqueName: \"kubernetes.io/projected/f25adc8f-59d7-4068-93f0-b3278c25e692-kube-api-access-gj4v4\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646263 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646152 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f25adc8f-59d7-4068-93f0-b3278c25e692-metrics-client-ca\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646263 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646446 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.646446 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.646315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747437 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747467 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-grpc-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747516 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747545 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747571 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4v4\" (UniqueName: \"kubernetes.io/projected/f25adc8f-59d7-4068-93f0-b3278c25e692-kube-api-access-gj4v4\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747790 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747640 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f25adc8f-59d7-4068-93f0-b3278c25e692-metrics-client-ca\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747790 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.747790 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.747750 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.748479 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.748425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f25adc8f-59d7-4068-93f0-b3278c25e692-metrics-client-ca\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750194 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750253 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750441 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750421 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750575 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750549 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-grpc-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750653 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750591 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-tls\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.750817 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.750800 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f25adc8f-59d7-4068-93f0-b3278c25e692-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.757508 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.757486 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4v4\" (UniqueName: \"kubernetes.io/projected/f25adc8f-59d7-4068-93f0-b3278c25e692-kube-api-access-gj4v4\") pod \"thanos-querier-69b765cb6-4nsxc\" (UID: \"f25adc8f-59d7-4068-93f0-b3278c25e692\") " pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:14.869173 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:14.869135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:15.077152 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.077126 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69b765cb6-4nsxc"] Jul 09 14:04:15.079258 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:15.079215 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25adc8f_59d7_4068_93f0_b3278c25e692.slice/crio-6dbd020fb0588dbd40b678050c35a4f7a853bbfeb36fcc932e95fae9623c9cfb WatchSource:0}: Error finding container 6dbd020fb0588dbd40b678050c35a4f7a853bbfeb36fcc932e95fae9623c9cfb: Status 404 returned error can't find the container with id 6dbd020fb0588dbd40b678050c35a4f7a853bbfeb36fcc932e95fae9623c9cfb Jul 09 14:04:15.398719 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.398683 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" event={"ID":"0deb68b5-ebd8-42f1-81b8-a9422077801d","Type":"ContainerStarted","Data":"086c5ae413ef76dea07b708ab6d8a93f0285d99c3b581d9334cda915dd2266e9"} Jul 09 14:04:15.399122 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.398743 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" event={"ID":"0deb68b5-ebd8-42f1-81b8-a9422077801d","Type":"ContainerStarted","Data":"d47fff9356609150a7d7f40964b66b8396cb6b2d9a3e8474f9605aa835461926"} Jul 09 14:04:15.399122 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.398771 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" event={"ID":"0deb68b5-ebd8-42f1-81b8-a9422077801d","Type":"ContainerStarted","Data":"088c7eb2a89fc40d8ed7df5b5a3661da533b837c5d10f87ed313fc6c8c63cc74"} Jul 09 14:04:15.400274 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.400249 2576 generic.go:356] "Generic (PLEG): container finished" podID="a17465a6-b481-4031-aa11-42699ff72664" containerID="4f4e3104589b54c57d93ab4c9bd914779c6bfff7ac523eb92df35bbbb61efdaa" exitCode=0 Jul 09 14:04:15.400419 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.400318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x6zvx" event={"ID":"a17465a6-b481-4031-aa11-42699ff72664","Type":"ContainerDied","Data":"4f4e3104589b54c57d93ab4c9bd914779c6bfff7ac523eb92df35bbbb61efdaa"} Jul 09 14:04:15.401564 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.401538 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"6dbd020fb0588dbd40b678050c35a4f7a853bbfeb36fcc932e95fae9623c9cfb"} Jul 09 14:04:15.423100 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:15.423053 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-9sc26" podStartSLOduration=1.606388014 podStartE2EDuration="3.423033357s" totalImagesPullingTime="1.816645343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:15.421510759 +0000 UTC m=+117.060185546" watchObservedRunningTime="2026-07-09 14:04:15.423033357 +0000 UTC m=+117.061708131" Jul 09 14:04:16.408572 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.408467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x6zvx" event={"ID":"a17465a6-b481-4031-aa11-42699ff72664","Type":"ContainerStarted","Data":"288b953c181a36c136f4e4d133d0d3ad0f2a43ef8c96800c0a35cdb331165810"} Jul 09 14:04:16.409045 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.408607 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x6zvx" event={"ID":"a17465a6-b481-4031-aa11-42699ff72664","Type":"ContainerStarted","Data":"94d869be6824f08abfc2f4209283916af70ebe8551a56e639e692c7cd51fb72d"} Jul 09 14:04:16.431639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.431590 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x6zvx" podStartSLOduration=2.992170201 podStartE2EDuration="4.431575067s" totalImagesPullingTime="1.344870571s" totalInitContainerRuntime="94.534295ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:16.430266183 +0000 UTC m=+118.068940975" watchObservedRunningTime="2026-07-09 14:04:16.431575067 +0000 UTC m=+118.070249839" Jul 09 14:04:16.859989 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.859950 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-798bd77ff7-wfg7b"] Jul 09 14:04:16.863621 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.863584 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.866574 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.866541 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 14:04:16.867897 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.867843 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 14:04:16.868025 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.868001 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rbq6r\"" Jul 09 14:04:16.868112 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.868044 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 14:04:16.868190 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.868176 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 14:04:16.868471 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.868280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c70egcqdakt9h\"" Jul 09 14:04:16.868977 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.868916 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798bd77ff7-wfg7b"] Jul 09 14:04:16.970620 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970586 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-client-certs\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970620 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970699 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-metrics-server-audit-profiles\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970721 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-tls\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970771 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-client-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970889 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970801 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62739d4a-d3fa-41d5-a923-1203f4e877e0-audit-log\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:16.970889 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:16.970847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwwf\" (UniqueName: \"kubernetes.io/projected/62739d4a-d3fa-41d5-a923-1203f4e877e0-kube-api-access-7cwwf\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071393 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62739d4a-d3fa-41d5-a923-1203f4e877e0-audit-log\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwwf\" (UniqueName: \"kubernetes.io/projected/62739d4a-d3fa-41d5-a923-1203f4e877e0-kube-api-access-7cwwf\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-client-certs\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-metrics-server-audit-profiles\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071759 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-tls\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.071759 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071588 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-client-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.073591 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.071749 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62739d4a-d3fa-41d5-a923-1203f4e877e0-audit-log\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.073591 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.072735 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-metrics-server-audit-profiles\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.073937 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.073874 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62739d4a-d3fa-41d5-a923-1203f4e877e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.074572 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.074504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-client-ca-bundle\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.074688 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.074673 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-client-certs\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.074688 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.074681 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62739d4a-d3fa-41d5-a923-1203f4e877e0-secret-metrics-server-tls\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.083022 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.082991 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwwf\" (UniqueName: \"kubernetes.io/projected/62739d4a-d3fa-41d5-a923-1203f4e877e0-kube-api-access-7cwwf\") pod \"metrics-server-798bd77ff7-wfg7b\" (UID: \"62739d4a-d3fa-41d5-a923-1203f4e877e0\") " pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.087219 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.087194 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:04:17.092455 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.092413 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:04:17.176284 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.176261 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:17.302961 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.302929 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g"] Jul 09 14:04:17.306027 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.306006 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.308895 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.308624 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 14:04:17.308895 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.308732 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qtqh7\"" Jul 09 14:04:17.314695 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.314673 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g"] Jul 09 14:04:17.325750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.325728 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798bd77ff7-wfg7b"] Jul 09 14:04:17.327434 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:17.327405 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62739d4a_d3fa_41d5_a923_1203f4e877e0.slice/crio-eb4ad2417151894e3d4ab4b4c2edc9ccd787c0e8fc61de20f7d3ce5b4d4dcced WatchSource:0}: Error finding container eb4ad2417151894e3d4ab4b4c2edc9ccd787c0e8fc61de20f7d3ce5b4d4dcced: Status 404 returned error can't find the container with id eb4ad2417151894e3d4ab4b4c2edc9ccd787c0e8fc61de20f7d3ce5b4d4dcced Jul 09 14:04:17.373689 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.373593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.373689 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.373653 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhwzg\" (UniqueName: \"kubernetes.io/projected/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-kube-api-access-qhwzg\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.414830 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.414797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"2dbfe5df87b6612afe807161bbad3f98b92211c8012bf12ee4bd4b1468bdf8e1"} Jul 09 14:04:17.415248 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.414872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"f98b533927667dcafc70cfe73dab450c003e394e6bfea183dfd2302a52829d1a"} Jul 09 14:04:17.415248 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.414914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"3e937f138357639eaa4c1e51d943f31946cd3f6190526d326c62c87ab5c00d34"} Jul 09 14:04:17.415968 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.415946 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" event={"ID":"62739d4a-d3fa-41d5-a923-1203f4e877e0","Type":"ContainerStarted","Data":"eb4ad2417151894e3d4ab4b4c2edc9ccd787c0e8fc61de20f7d3ce5b4d4dcced"} Jul 09 14:04:17.474700 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.474661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.474879 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.474777 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhwzg\" (UniqueName: \"kubernetes.io/projected/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-kube-api-access-qhwzg\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.474947 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:17.474887 2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 09 14:04:17.474993 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:17.474961 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert podName:dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209 nodeName:}" failed. No retries permitted until 2026-07-09 14:04:17.97494035 +0000 UTC m=+119.613615102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-7r27g" (UID: "dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209") : secret "monitoring-plugin-cert" not found Jul 09 14:04:17.483182 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.483163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhwzg\" (UniqueName: \"kubernetes.io/projected/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-kube-api-access-qhwzg\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.980067 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:17.979981 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:17.980347 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:17.980154 2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 09 14:04:17.980347 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:17.980250 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert podName:dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209 nodeName:}" failed. No retries permitted until 2026-07-09 14:04:18.980213375 +0000 UTC m=+120.618888127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-7r27g" (UID: "dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209") : secret "monitoring-plugin-cert" not found Jul 09 14:04:18.427600 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.427558 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"2817475b25ab283a0c1cf54a29d69300f90b8a255e69580e96dbde30b4761a18"} Jul 09 14:04:18.428202 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.427650 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"e63a59b1a571b060f72a90267c46f7d291c0620facbc6c1f50833fe656e7affc"} Jul 09 14:04:18.428202 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.427686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" event={"ID":"f25adc8f-59d7-4068-93f0-b3278c25e692","Type":"ContainerStarted","Data":"01e7fbec7ebd8ca803bc20df1c695cac13794fa4ec96cd34c16aa168c301ae68"} Jul 09 14:04:18.428202 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.427792 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:18.455004 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.453901 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" podStartSLOduration=1.553666309 podStartE2EDuration="4.453880037s" totalImagesPullingTime="2.900213728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:18.450746032 +0000 UTC m=+120.089420861" watchObservedRunningTime="2026-07-09 14:04:18.453880037 +0000 UTC m=+120.092554816" Jul 09 14:04:18.990665 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.990638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:18.993210 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:18.993187 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-7r27g\" (UID: \"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:19.120279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.120221 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qtqh7\"" Jul 09 14:04:19.128482 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.128458 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:19.275854 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.275763 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g"] Jul 09 14:04:19.280212 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:19.280183 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfd7a6b_0c44_4d11_8eb8_6f5d7ebfd209.slice/crio-fa66608c3a9a31d080ac6a521f4dfe0e378aaf2e257656cbd63f61d46cb7b969 WatchSource:0}: Error finding container fa66608c3a9a31d080ac6a521f4dfe0e378aaf2e257656cbd63f61d46cb7b969: Status 404 returned error can't find the container with id fa66608c3a9a31d080ac6a521f4dfe0e378aaf2e257656cbd63f61d46cb7b969 Jul 09 14:04:19.432996 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.432955 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" event={"ID":"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209","Type":"ContainerStarted","Data":"fa66608c3a9a31d080ac6a521f4dfe0e378aaf2e257656cbd63f61d46cb7b969"} Jul 09 14:04:19.434502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.434460 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" event={"ID":"62739d4a-d3fa-41d5-a923-1203f4e877e0","Type":"ContainerStarted","Data":"7a996a1d18a70dbc8bf7d9e59028069b7a215d69f9301944143a52b857c25725"} Jul 09 14:04:19.454465 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:19.454406 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" podStartSLOduration=1.806146584 podStartE2EDuration="3.454390844s" totalImagesPullingTime="1.64824426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:19.453893997 +0000 UTC m=+121.092568797" watchObservedRunningTime="2026-07-09 14:04:19.454390844 +0000 UTC m=+121.093065617" Jul 09 14:04:21.444312 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:21.444265 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" event={"ID":"dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209","Type":"ContainerStarted","Data":"a97f244663d1246b7697da91f508526bfbfcb215946e7b5839097d6ba5e44fea"} Jul 09 14:04:21.444694 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:21.444341 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:21.449130 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:21.449106 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" Jul 09 14:04:21.461208 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:21.461164 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-7r27g" podStartSLOduration=2.46718531 podStartE2EDuration="4.461148796s" totalImagesPullingTime="1.993963486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:21.459519893 +0000 UTC m=+123.098194696" watchObservedRunningTime="2026-07-09 14:04:21.461148796 +0000 UTC m=+123.099823570" Jul 09 14:04:23.389940 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.389892 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:04:23.395109 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.395081 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.400869 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.400836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 14:04:23.400869 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.400865 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 14:04:23.401645 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.401315 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 14:04:23.401645 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.401349 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 14:04:23.401645 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.401370 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7f2kz\"" Jul 09 14:04:23.402269 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.402245 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 14:04:23.402269 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.402258 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 14:04:23.402269 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.402223 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 14:04:23.414188 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.414161 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:04:23.531960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.531914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.532136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.531974 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.532136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.532056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.532136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.532105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.532136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.532126 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.532307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.532154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdkt8\" (UniqueName: \"kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633357 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633407 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633634 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633451 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633634 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633634 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.633634 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.633532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdkt8\" (UniqueName: \"kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.634270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.634215 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.634365 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.634272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.634365 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.634303 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.635914 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.635892 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.635961 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.635910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.641868 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.641811 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdkt8\" (UniqueName: \"kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8\") pod \"console-658d674fcd-w52xv\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.703993 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.703951 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:23.846398 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:23.846363 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:04:23.850007 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:23.849984 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae9f64d_46c3_4b57_961e_45dec546ea8e.slice/crio-895b4a6ee8a45d8ecdee0b6cfe4cfb4621180b224645ba6d278d15664d047228 WatchSource:0}: Error finding container 895b4a6ee8a45d8ecdee0b6cfe4cfb4621180b224645ba6d278d15664d047228: Status 404 returned error can't find the container with id 895b4a6ee8a45d8ecdee0b6cfe4cfb4621180b224645ba6d278d15664d047228 Jul 09 14:04:24.441926 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:24.441877 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-69b765cb6-4nsxc" Jul 09 14:04:24.457409 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:24.457374 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658d674fcd-w52xv" event={"ID":"6ae9f64d-46c3-4b57-961e-45dec546ea8e","Type":"ContainerStarted","Data":"895b4a6ee8a45d8ecdee0b6cfe4cfb4621180b224645ba6d278d15664d047228"} Jul 09 14:04:27.471776 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.471687 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658d674fcd-w52xv" event={"ID":"6ae9f64d-46c3-4b57-961e-45dec546ea8e","Type":"ContainerStarted","Data":"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d"} Jul 09 14:04:27.490535 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.490480 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658d674fcd-w52xv" podStartSLOduration=1.202661286 podStartE2EDuration="4.490457362s" totalImagesPullingTime="3.287796076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:27.489785202 +0000 UTC m=+129.128459976" watchObservedRunningTime="2026-07-09 14:04:27.490457362 +0000 UTC m=+129.129132137" Jul 09 14:04:27.768097 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.768002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:04:27.770336 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.770311 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c96d14-5339-496a-8894-e4aeef01e20c-metrics-certs\") pod \"network-metrics-daemon-tvvs6\" (UID: \"47c96d14-5339-496a-8894-e4aeef01e20c\") " pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:04:27.808705 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.808670 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27nh2\"" Jul 09 14:04:27.815991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.815963 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvvs6" Jul 09 14:04:27.950026 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:27.949998 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvvs6"] Jul 09 14:04:27.952587 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:27.952557 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c96d14_5339_496a_8894_e4aeef01e20c.slice/crio-5f5814fc51023e95a03885a336bbaaf30447bc2f711f6fd879822d0a54e5cf31 WatchSource:0}: Error finding container 5f5814fc51023e95a03885a336bbaaf30447bc2f711f6fd879822d0a54e5cf31: Status 404 returned error can't find the container with id 5f5814fc51023e95a03885a336bbaaf30447bc2f711f6fd879822d0a54e5cf31 Jul 09 14:04:28.477092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:28.477061 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvvs6" event={"ID":"47c96d14-5339-496a-8894-e4aeef01e20c","Type":"ContainerStarted","Data":"5f5814fc51023e95a03885a336bbaaf30447bc2f711f6fd879822d0a54e5cf31"} Jul 09 14:04:29.483303 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:29.483263 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvvs6" event={"ID":"47c96d14-5339-496a-8894-e4aeef01e20c","Type":"ContainerStarted","Data":"c2a6b0144eca56c90754cac9c023e3394eef9c0b7e03d3d685d16157ddbb2acc"} Jul 09 14:04:29.483884 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:29.483326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvvs6" event={"ID":"47c96d14-5339-496a-8894-e4aeef01e20c","Type":"ContainerStarted","Data":"94abdc48813d0f541e0237f811b4898aee19c069e779fd1d8c9b694d01e5ebc4"} Jul 09 14:04:29.502284 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:29.502216 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tvvs6" podStartSLOduration=129.403361429 podStartE2EDuration="2m10.502198815s" totalImagesPullingTime="1.098837386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:29.500328474 +0000 UTC m=+131.139003275" watchObservedRunningTime="2026-07-09 14:04:29.502198815 +0000 UTC m=+131.140873588" Jul 09 14:04:31.179185 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.179147 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:04:31.182261 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.182219 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.195726 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.195699 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 14:04:31.197270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.197245 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:04:31.298003 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.297969 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298099 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298326 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298326 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298272 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298326 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298305 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.298423 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.298332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmfxv\" (UniqueName: \"kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.398852 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.398799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.398852 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.398853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399106 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.398969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmfxv\" (UniqueName: \"kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399106 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399106 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399293 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399293 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399179 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399631 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399605 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.399953 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399929 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.400015 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.399941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.401484 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.401465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.401556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.401494 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.413525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.413499 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmfxv\" (UniqueName: \"kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv\") pod \"console-7f4849f648-9pxmt\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.492783 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.492703 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:31.629005 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:31.628975 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:04:31.630876 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:04:31.630847 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1797f7cf_e20a_4121_ba36_a22cb46feff2.slice/crio-1e4c537c49df946298fa0e1f9d1913525310d39add1ee2b9ca7e13c3bad1b6e0 WatchSource:0}: Error finding container 1e4c537c49df946298fa0e1f9d1913525310d39add1ee2b9ca7e13c3bad1b6e0: Status 404 returned error can't find the container with id 1e4c537c49df946298fa0e1f9d1913525310d39add1ee2b9ca7e13c3bad1b6e0 Jul 09 14:04:32.496996 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:32.496960 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4849f648-9pxmt" event={"ID":"1797f7cf-e20a-4121-ba36-a22cb46feff2","Type":"ContainerStarted","Data":"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141"} Jul 09 14:04:32.497387 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:32.497009 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4849f648-9pxmt" event={"ID":"1797f7cf-e20a-4121-ba36-a22cb46feff2","Type":"ContainerStarted","Data":"1e4c537c49df946298fa0e1f9d1913525310d39add1ee2b9ca7e13c3bad1b6e0"} Jul 09 14:04:32.516747 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:32.516688 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f4849f648-9pxmt" podStartSLOduration=1.5166704690000001 podStartE2EDuration="1.516670469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:32.514855748 +0000 UTC m=+134.153530537" watchObservedRunningTime="2026-07-09 14:04:32.516670469 +0000 UTC m=+134.155345243" Jul 09 14:04:33.704927 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:33.704881 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:33.705419 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:33.704956 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:33.709960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:33.709935 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:34.508773 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:34.508742 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:04:37.177300 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:37.177259 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:37.177675 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:37.177329 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:38.522204 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:38.522167 2576 generic.go:356] "Generic (PLEG): container finished" podID="7a47e50e-4829-4881-bea1-bbeebe26be21" containerID="b8b252b6435c1b2738607e3e2834433752dfed3d074c0521faf12b4ad51b9435" exitCode=0 Jul 09 14:04:38.522637 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:38.522224 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" event={"ID":"7a47e50e-4829-4881-bea1-bbeebe26be21","Type":"ContainerDied","Data":"b8b252b6435c1b2738607e3e2834433752dfed3d074c0521faf12b4ad51b9435"} Jul 09 14:04:38.522637 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:38.522526 2576 scope.go:117] "RemoveContainer" containerID="b8b252b6435c1b2738607e3e2834433752dfed3d074c0521faf12b4ad51b9435" Jul 09 14:04:39.529569 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:39.529524 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m5j9w" event={"ID":"7a47e50e-4829-4881-bea1-bbeebe26be21","Type":"ContainerStarted","Data":"f21d2e285ec2767c4f4eb69ffc919b14cdd15881b4d08a7b89533f70e5cbe00e"} Jul 09 14:04:40.536368 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:40.536330 2576 generic.go:356] "Generic (PLEG): container finished" podID="9243e7ec-aa19-4e2c-adeb-18debfd84b6f" containerID="22ec3235fffcd269c4496a837b852767d7179d8328fa5ef1cd9c225486f4dabc" exitCode=0 Jul 09 14:04:40.536744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:40.536401 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" event={"ID":"9243e7ec-aa19-4e2c-adeb-18debfd84b6f","Type":"ContainerDied","Data":"22ec3235fffcd269c4496a837b852767d7179d8328fa5ef1cd9c225486f4dabc"} Jul 09 14:04:40.536787 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:40.536764 2576 scope.go:117] "RemoveContainer" containerID="22ec3235fffcd269c4496a837b852767d7179d8328fa5ef1cd9c225486f4dabc" Jul 09 14:04:41.493728 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.493692 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:41.493916 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.493760 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:41.499902 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.499876 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:41.547031 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.546996 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p9xm2" event={"ID":"9243e7ec-aa19-4e2c-adeb-18debfd84b6f","Type":"ContainerStarted","Data":"df400adad354569fd66cc78a136f682ac31870591b9e3474dc1e26e39daa0cf8"} Jul 09 14:04:41.554436 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.552828 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:04:41.619874 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:41.619831 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:04:42.113947 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.113895 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" podUID="edad103a-e449-4533-a278-54aa63dd07d6" containerName="registry" containerID="cri-o://bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7" gracePeriod=30 Jul 09 14:04:42.411763 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.411726 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:04:42.550845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.550816 2576 generic.go:356] "Generic (PLEG): container finished" podID="edad103a-e449-4533-a278-54aa63dd07d6" containerID="bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7" exitCode=0 Jul 09 14:04:42.551291 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.550874 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" Jul 09 14:04:42.551291 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.550901 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" event={"ID":"edad103a-e449-4533-a278-54aa63dd07d6","Type":"ContainerDied","Data":"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7"} Jul 09 14:04:42.551291 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.550953 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bbf4cb7c-v58tr" event={"ID":"edad103a-e449-4533-a278-54aa63dd07d6","Type":"ContainerDied","Data":"35ffdc90e050a09484daafa8066b2d17c23423735586eb0d679c10d514a24fb3"} Jul 09 14:04:42.551291 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.550976 2576 scope.go:117] "RemoveContainer" containerID="bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7" Jul 09 14:04:42.559466 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.559440 2576 scope.go:117] "RemoveContainer" containerID="bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7" Jul 09 14:04:42.559814 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:04:42.559782 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7\": container with ID starting with bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7 not found: ID does not exist" containerID="bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7" Jul 09 14:04:42.559900 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.559821 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7"} err="failed to get container status \"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7\": rpc error: code = NotFound desc = could not find container \"bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7\": container with ID starting with bb8ad45d5d0f9e3d40acd3a57f74c5c4f691653f068cbdd6cb2771a14f7daca7 not found: ID does not exist" Jul 09 14:04:42.594165 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594126 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594336 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594214 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594336 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594316 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5qcb\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594415 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594367 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594522 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594415 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594522 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594443 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594522 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594473 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594522 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594496 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates\") pod \"edad103a-e449-4533-a278-54aa63dd07d6\" (UID: \"edad103a-e449-4533-a278-54aa63dd07d6\") " Jul 09 14:04:42.594721 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594556 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:04:42.594797 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.594769 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-trusted-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.595146 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.595105 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:04:42.596833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.596805 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:04:42.596961 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.596939 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb" (OuterVolumeSpecName: "kube-api-access-q5qcb") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "kube-api-access-q5qcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:42.597034 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.597009 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:42.597089 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.597014 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:42.597167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.597143 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:04:42.603673 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.603645 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "edad103a-e449-4533-a278-54aa63dd07d6" (UID: "edad103a-e449-4533-a278-54aa63dd07d6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695284 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-bound-sa-token\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695326 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5qcb\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-kube-api-access-q5qcb\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695340 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-installation-pull-secrets\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695354 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edad103a-e449-4533-a278-54aa63dd07d6-ca-trust-extracted\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695369 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edad103a-e449-4533-a278-54aa63dd07d6-registry-tls\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695386 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edad103a-e449-4533-a278-54aa63dd07d6-image-registry-private-configuration\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.695534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.695400 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edad103a-e449-4533-a278-54aa63dd07d6-registry-certificates\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:04:42.873357 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.873321 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:04:42.879195 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.879163 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86bbf4cb7c-v58tr"] Jul 09 14:04:42.893746 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:42.893719 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edad103a-e449-4533-a278-54aa63dd07d6" path="/var/lib/kubelet/pods/edad103a-e449-4533-a278-54aa63dd07d6/volumes" Jul 09 14:04:43.968129 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:43.968093 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-nwltl_205237ba-20da-4c38-8008-9a4403f17b1f/cluster-monitoring-operator/0.log" Jul 09 14:04:44.167152 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:44.167119 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-state-metrics/0.log" Jul 09 14:04:44.365115 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:44.365082 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-rbac-proxy-main/0.log" Jul 09 14:04:44.565015 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:44.564986 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-rbac-proxy-self/0.log" Jul 09 14:04:44.765381 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:44.765295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-798bd77ff7-wfg7b_62739d4a-d3fa-41d5-a923-1203f4e877e0/metrics-server/0.log" Jul 09 14:04:44.966962 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:44.966927 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-7r27g_dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209/monitoring-plugin/0.log" Jul 09 14:04:45.765567 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:45.765535 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/init-textfile/0.log" Jul 09 14:04:45.966812 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:45.966777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/node-exporter/0.log" Jul 09 14:04:46.174116 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:46.174081 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/kube-rbac-proxy/0.log" Jul 09 14:04:49.565314 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.565284 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/thanos-query/0.log" Jul 09 14:04:49.580609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.580578 2576 generic.go:356] "Generic (PLEG): container finished" podID="050a74e3-9444-46b7-863e-17c6b5148bad" containerID="a3de11b68b56cc8ecb068f87423e1c657dafb92ed7b680adc4ec822d609daedc" exitCode=0 Jul 09 14:04:49.580756 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.580654 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" event={"ID":"050a74e3-9444-46b7-863e-17c6b5148bad","Type":"ContainerDied","Data":"a3de11b68b56cc8ecb068f87423e1c657dafb92ed7b680adc4ec822d609daedc"} Jul 09 14:04:49.580989 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.580976 2576 scope.go:117] "RemoveContainer" containerID="a3de11b68b56cc8ecb068f87423e1c657dafb92ed7b680adc4ec822d609daedc" Jul 09 14:04:49.764309 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.764275 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-web/0.log" Jul 09 14:04:49.965310 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:49.965210 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy/0.log" Jul 09 14:04:50.167009 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.166976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/prom-label-proxy/0.log" Jul 09 14:04:50.365358 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.365327 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-rules/0.log" Jul 09 14:04:50.565720 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.565675 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-metrics/0.log" Jul 09 14:04:50.587416 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.587382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mxxwh" event={"ID":"050a74e3-9444-46b7-863e-17c6b5148bad","Type":"ContainerStarted","Data":"75d455ac5f2e13d2886764488eddb91117f057b70d8703be53fe301494b6cb35"} Jul 09 14:04:50.768764 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.768640 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-pw5d6_2b62edca-81e3-4de9-88e0-bd08cd290874/networking-console-plugin/0.log" Jul 09 14:04:50.965095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:50.965062 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:04:51.168633 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:51.168581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/3.log" Jul 09 14:04:51.366326 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:51.366292 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-658d674fcd-w52xv_6ae9f64d-46c3-4b57-961e-45dec546ea8e/console/0.log" Jul 09 14:04:51.565562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:51.565525 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f4849f648-9pxmt_1797f7cf-e20a-4121-ba36-a22cb46feff2/console/0.log" Jul 09 14:04:51.967905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:51.967826 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bb56f78d-q42xs_0dece533-f971-48c8-972c-c312fe875473/router/0.log" Jul 09 14:04:52.565461 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:52.565433 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jczw6_c302bd4d-03dc-48a3-aa09-0e14dfc592d0/serve-healthcheck-canary/0.log" Jul 09 14:04:57.182480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:57.182448 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:04:57.186341 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:04:57.186313 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-798bd77ff7-wfg7b" Jul 09 14:05:06.642543 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:06.642473 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-658d674fcd-w52xv" podUID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" containerName="console" containerID="cri-o://e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d" gracePeriod=15 Jul 09 14:05:06.897260 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:06.897183 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:05:07.009048 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009008 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009255 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009085 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009255 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009123 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009255 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009144 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009255 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009176 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdkt8\" (UniqueName: \"kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009473 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009301 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert\") pod \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\" (UID: \"6ae9f64d-46c3-4b57-961e-45dec546ea8e\") " Jul 09 14:05:07.009527 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009499 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:05:07.009527 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009514 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config" (OuterVolumeSpecName: "console-config") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:05:07.009631 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009588 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:05:07.009683 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009623 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-oauth-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.009683 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.009653 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.011513 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.011478 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:05:07.011513 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.011497 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:05:07.011650 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.011561 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8" (OuterVolumeSpecName: "kube-api-access-rdkt8") pod "6ae9f64d-46c3-4b57-961e-45dec546ea8e" (UID: "6ae9f64d-46c3-4b57-961e-45dec546ea8e"). InnerVolumeSpecName "kube-api-access-rdkt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:05:07.110447 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.110400 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae9f64d-46c3-4b57-961e-45dec546ea8e-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.110447 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.110438 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-oauth-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.110447 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.110452 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdkt8\" (UniqueName: \"kubernetes.io/projected/6ae9f64d-46c3-4b57-961e-45dec546ea8e-kube-api-access-rdkt8\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.110682 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.110465 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ae9f64d-46c3-4b57-961e-45dec546ea8e-console-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:05:07.665394 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.665360 2576 generic.go:356] "Generic (PLEG): container finished" podID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" containerID="e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d" exitCode=0 Jul 09 14:05:07.665848 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.665404 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658d674fcd-w52xv" Jul 09 14:05:07.665848 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.665442 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658d674fcd-w52xv" event={"ID":"6ae9f64d-46c3-4b57-961e-45dec546ea8e","Type":"ContainerDied","Data":"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d"} Jul 09 14:05:07.665848 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.665503 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658d674fcd-w52xv" event={"ID":"6ae9f64d-46c3-4b57-961e-45dec546ea8e","Type":"ContainerDied","Data":"895b4a6ee8a45d8ecdee0b6cfe4cfb4621180b224645ba6d278d15664d047228"} Jul 09 14:05:07.665848 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.665535 2576 scope.go:117] "RemoveContainer" containerID="e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d" Jul 09 14:05:07.674990 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.674969 2576 scope.go:117] "RemoveContainer" containerID="e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d" Jul 09 14:05:07.675296 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:05:07.675273 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d\": container with ID starting with e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d not found: ID does not exist" containerID="e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d" Jul 09 14:05:07.675354 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.675307 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d"} err="failed to get container status \"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d\": rpc error: code = NotFound desc = could not find container \"e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d\": container with ID starting with e570ebb9a5536fe798e63f743a420dbdd3e83492a90a3bf496fd42b543bef44d not found: ID does not exist" Jul 09 14:05:07.688216 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.688187 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:05:07.691378 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:07.691347 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658d674fcd-w52xv"] Jul 09 14:05:08.894048 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:08.894011 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" path="/var/lib/kubelet/pods/6ae9f64d-46c3-4b57-961e-45dec546ea8e/volumes" Jul 09 14:05:34.570710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.570670 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571131 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" containerName="console" Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571151 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" containerName="console" Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571168 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edad103a-e449-4533-a278-54aa63dd07d6" containerName="registry" Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571177 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="edad103a-e449-4533-a278-54aa63dd07d6" containerName="registry" Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571294 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="edad103a-e449-4533-a278-54aa63dd07d6" containerName="registry" Jul 09 14:05:34.571331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.571309 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae9f64d-46c3-4b57-961e-45dec546ea8e" containerName="console" Jul 09 14:05:34.576344 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.576322 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.583024 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583004 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 14:05:34.583364 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583295 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 14:05:34.583364 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583319 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 14:05:34.583364 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bgjks\"" Jul 09 14:05:34.583364 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583360 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 14:05:34.583583 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583359 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 14:05:34.583628 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583612 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 14:05:34.583666 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583638 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 14:05:34.583709 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.583638 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 14:05:34.590508 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.590482 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 14:05:34.597532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.597506 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:05:34.665114 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665078 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665114 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665117 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-config-volume\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-web-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665327 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665364 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-config-out\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665465 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665495 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb82z\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-kube-api-access-jb82z\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.665585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.665571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766622 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766664 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-web-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766914 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766681 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766914 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-config-out\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766914 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.766914 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766908 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766938 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.766994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767046 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb82z\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-kube-api-access-jb82z\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-config-volume\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767505 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767341 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.767632 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.767597 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.768747 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.768413 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d022a56d-d3ab-482f-b1a5-491647894c01-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.769980 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.769863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.769980 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.769930 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770143 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770051 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-config-volume\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770143 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d022a56d-d3ab-482f-b1a5-491647894c01-config-out\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770557 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770701 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770622 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-web-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770701 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770675 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.770960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.770943 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.772149 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.772127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d022a56d-d3ab-482f-b1a5-491647894c01-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.781731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.781709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb82z\" (UniqueName: \"kubernetes.io/projected/d022a56d-d3ab-482f-b1a5-491647894c01-kube-api-access-jb82z\") pod \"alertmanager-main-0\" (UID: \"d022a56d-d3ab-482f-b1a5-491647894c01\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:34.885178 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:34.885097 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:05:35.042268 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:35.042243 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:05:35.044190 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:05:35.044164 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd022a56d_d3ab_482f_b1a5_491647894c01.slice/crio-1f2c00d4da94806ead627336f5aa70b0fdb4f129842eb7059fad711a325b31bd WatchSource:0}: Error finding container 1f2c00d4da94806ead627336f5aa70b0fdb4f129842eb7059fad711a325b31bd: Status 404 returned error can't find the container with id 1f2c00d4da94806ead627336f5aa70b0fdb4f129842eb7059fad711a325b31bd Jul 09 14:05:35.779076 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:35.779042 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"1f2c00d4da94806ead627336f5aa70b0fdb4f129842eb7059fad711a325b31bd"} Jul 09 14:05:36.784507 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.784473 2576 generic.go:356] "Generic (PLEG): container finished" podID="d022a56d-d3ab-482f-b1a5-491647894c01" containerID="baf842c5fe99d4d870589b3e32a83e8c607627b344e36f35215b20b5976eb8f2" exitCode=0 Jul 09 14:05:36.784980 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.784575 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerDied","Data":"baf842c5fe99d4d870589b3e32a83e8c607627b344e36f35215b20b5976eb8f2"} Jul 09 14:05:36.809657 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.809626 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5778d8b68f-9bqch"] Jul 09 14:05:36.813875 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.813849 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.817361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.817334 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 14:05:36.817508 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.817487 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 14:05:36.817560 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.817549 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w9b87\"" Jul 09 14:05:36.817852 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.817836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 14:05:36.817900 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.817871 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 14:05:36.818194 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.818174 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 14:05:36.822731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.822714 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 14:05:36.830010 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.829989 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5778d8b68f-9bqch"] Jul 09 14:05:36.884374 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-federate-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884374 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-serving-certs-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884526 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884644 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-metrics-client-ca\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.884836 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.884742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.885107 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.885080 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn74t\" (UniqueName: \"kubernetes.io/projected/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-kube-api-access-sn74t\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986460 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986422 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn74t\" (UniqueName: \"kubernetes.io/projected/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-kube-api-access-sn74t\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-federate-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986527 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-serving-certs-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986542 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986577 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-metrics-client-ca\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986602 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.986906 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.986632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.987538 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.987434 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-serving-certs-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.987538 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.987487 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-metrics-client-ca\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.987683 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.987540 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.989370 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.989340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-federate-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.989473 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.989376 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.989515 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.989476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-telemeter-client-tls\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.989551 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.989519 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:36.995222 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:36.995203 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn74t\" (UniqueName: \"kubernetes.io/projected/b4023edb-d68c-458a-8fb1-fb7588f3c6b9-kube-api-access-sn74t\") pod \"telemeter-client-5778d8b68f-9bqch\" (UID: \"b4023edb-d68c-458a-8fb1-fb7588f3c6b9\") " pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:37.126022 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:37.125981 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" Jul 09 14:05:37.270148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:37.270108 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5778d8b68f-9bqch"] Jul 09 14:05:37.274541 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:05:37.274508 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4023edb_d68c_458a_8fb1_fb7588f3c6b9.slice/crio-5ae9e3530e1b93045eddcdcfd0f01d769db3c0bf7de9f27589310bb14dcd58ab WatchSource:0}: Error finding container 5ae9e3530e1b93045eddcdcfd0f01d769db3c0bf7de9f27589310bb14dcd58ab: Status 404 returned error can't find the container with id 5ae9e3530e1b93045eddcdcfd0f01d769db3c0bf7de9f27589310bb14dcd58ab Jul 09 14:05:37.790190 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:37.790156 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" event={"ID":"b4023edb-d68c-458a-8fb1-fb7588f3c6b9","Type":"ContainerStarted","Data":"5ae9e3530e1b93045eddcdcfd0f01d769db3c0bf7de9f27589310bb14dcd58ab"} Jul 09 14:05:38.796938 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:38.796909 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"d6605761c71bb3ce3d22ef86ce65ce350d7869313e23b157f62f37e457f2988b"} Jul 09 14:05:38.797225 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:38.796972 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"b66027fb5c7bde116c346a10fd620421ba43ddd5a40af2e07b41d1d0074ee739"} Jul 09 14:05:39.804690 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.804591 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"bbf2197684d7ded51134e0cbeda3781ac4e950c7795a4e31bed0d3faf0281f3d"} Jul 09 14:05:39.804690 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.804669 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"4e961407c0dec10dce1ad65bdef7d095392feb709aacbc64db32a0fe57610e14"} Jul 09 14:05:39.805270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.804707 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"b2211b652ebd4eb7cf01176827bf940a352f52fc8ee61dbae89f0cdad3bcb44f"} Jul 09 14:05:39.805270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.804746 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d022a56d-d3ab-482f-b1a5-491647894c01","Type":"ContainerStarted","Data":"11df4f6982b463b70f44b320047b4351b76e6576a3d84eaa425291ab85c87434"} Jul 09 14:05:39.806618 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.806580 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" event={"ID":"b4023edb-d68c-458a-8fb1-fb7588f3c6b9","Type":"ContainerStarted","Data":"9b18cb6d5dcd7912982dcd0f645fc319f8042822dbbc989e958a5a0dbe79b227"} Jul 09 14:05:39.806731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.806630 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" event={"ID":"b4023edb-d68c-458a-8fb1-fb7588f3c6b9","Type":"ContainerStarted","Data":"87ac8db9435b06c34e93ba237084fa0dfef6336768e3af77b75740ccd26134b4"} Jul 09 14:05:39.806731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.806648 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" event={"ID":"b4023edb-d68c-458a-8fb1-fb7588f3c6b9","Type":"ContainerStarted","Data":"135aab1c6e9be3d5898125d70cbee8eb8540448f3f930f49d5ee6d4c0990f4a2"} Jul 09 14:05:39.835958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.835904 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.835369822 podStartE2EDuration="5.835885626s" totalImagesPullingTime="2.984873367s" totalInitContainerRuntime="15.642437ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:05:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:05:39.834863661 +0000 UTC m=+201.473538493" watchObservedRunningTime="2026-07-09 14:05:39.835885626 +0000 UTC m=+201.474560450" Jul 09 14:05:39.857191 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:39.857137 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5778d8b68f-9bqch" podStartSLOduration=1.622164825 podStartE2EDuration="3.85711954s" totalImagesPullingTime="2.234954715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:05:39.85537375 +0000 UTC m=+201.494048551" watchObservedRunningTime="2026-07-09 14:05:39.85711954 +0000 UTC m=+201.495794333" Jul 09 14:05:40.899965 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.899917 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:05:40.904348 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.904319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.915927 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.915894 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:05:40.922082 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922318 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922196 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922435 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922316 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922435 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922382 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922435 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmqz\" (UniqueName: \"kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922643 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922457 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:40.922643 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:40.922629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.023833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023789 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.023833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023861 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023880 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqz\" (UniqueName: \"kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.023942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.024696 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024724 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.024713 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024888 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.024695 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.024984 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.024962 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.026443 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.026421 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.026534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.026512 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.031810 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.031793 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmqz\" (UniqueName: \"kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz\") pod \"console-846b67c96f-z5ldg\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.215592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.215509 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:41.363840 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.360956 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:05:41.820283 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.820226 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846b67c96f-z5ldg" event={"ID":"a0ba67ae-7728-403d-902d-f0d273331803","Type":"ContainerStarted","Data":"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08"} Jul 09 14:05:41.820502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.820306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846b67c96f-z5ldg" event={"ID":"a0ba67ae-7728-403d-902d-f0d273331803","Type":"ContainerStarted","Data":"e1a79427f1607cff32e8188cde2ab521d45ce2fb0b7f13980cefac666d180c1e"} Jul 09 14:05:41.839300 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:41.839249 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-846b67c96f-z5ldg" podStartSLOduration=1.839214504 podStartE2EDuration="1.839214504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:05:41.83914682 +0000 UTC m=+203.477821650" watchObservedRunningTime="2026-07-09 14:05:41.839214504 +0000 UTC m=+203.477889278" Jul 09 14:05:51.216010 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:51.215977 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:51.216431 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:51.216029 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:51.220804 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:51.220779 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:51.867085 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:51.867050 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:05:51.913862 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:05:51.913829 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:06:16.942495 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:16.942435 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f4849f648-9pxmt" podUID="1797f7cf-e20a-4121-ba36-a22cb46feff2" containerName="console" containerID="cri-o://9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141" gracePeriod=15 Jul 09 14:06:17.190490 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.190465 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:06:17.354121 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354086 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354121 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354127 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354397 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354148 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354397 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354175 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354397 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354195 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmfxv\" (UniqueName: \"kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354397 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354220 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354397 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354301 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert\") pod \"1797f7cf-e20a-4121-ba36-a22cb46feff2\" (UID: \"1797f7cf-e20a-4121-ba36-a22cb46feff2\") " Jul 09 14:06:17.354853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354810 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca" (OuterVolumeSpecName: "service-ca") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:17.354959 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354824 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config" (OuterVolumeSpecName: "console-config") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:17.354959 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.354583 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:17.355108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.355071 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-trusted-ca-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.355108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.355106 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.355108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.355118 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.355472 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.355444 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:17.356632 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.356602 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv" (OuterVolumeSpecName: "kube-api-access-wmfxv") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "kube-api-access-wmfxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:17.356845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.356825 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:06:17.356909 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.356846 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1797f7cf-e20a-4121-ba36-a22cb46feff2" (UID: "1797f7cf-e20a-4121-ba36-a22cb46feff2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:06:17.455497 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.455459 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1797f7cf-e20a-4121-ba36-a22cb46feff2-oauth-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.455497 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.455490 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.455497 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.455499 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1797f7cf-e20a-4121-ba36-a22cb46feff2-console-oauth-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.455725 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.455507 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmfxv\" (UniqueName: \"kubernetes.io/projected/1797f7cf-e20a-4121-ba36-a22cb46feff2-kube-api-access-wmfxv\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:06:17.985223 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.985188 2576 generic.go:356] "Generic (PLEG): container finished" podID="1797f7cf-e20a-4121-ba36-a22cb46feff2" containerID="9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141" exitCode=0 Jul 09 14:06:17.985635 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.985275 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4849f648-9pxmt" Jul 09 14:06:17.985635 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.985279 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4849f648-9pxmt" event={"ID":"1797f7cf-e20a-4121-ba36-a22cb46feff2","Type":"ContainerDied","Data":"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141"} Jul 09 14:06:17.985635 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.985333 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4849f648-9pxmt" event={"ID":"1797f7cf-e20a-4121-ba36-a22cb46feff2","Type":"ContainerDied","Data":"1e4c537c49df946298fa0e1f9d1913525310d39add1ee2b9ca7e13c3bad1b6e0"} Jul 09 14:06:17.985635 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.985353 2576 scope.go:117] "RemoveContainer" containerID="9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141" Jul 09 14:06:17.994792 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.994767 2576 scope.go:117] "RemoveContainer" containerID="9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141" Jul 09 14:06:17.995066 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:06:17.995044 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141\": container with ID starting with 9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141 not found: ID does not exist" containerID="9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141" Jul 09 14:06:17.995131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:17.995078 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141"} err="failed to get container status \"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141\": rpc error: code = NotFound desc = could not find container \"9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141\": container with ID starting with 9508da10bd1e7b8517872ff65f7a15af3cc3d2b549244564b43136e6c8a37141 not found: ID does not exist" Jul 09 14:06:18.007280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:18.007248 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:06:18.011365 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:18.011334 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f4849f648-9pxmt"] Jul 09 14:06:18.894561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:06:18.894526 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1797f7cf-e20a-4121-ba36-a22cb46feff2" path="/var/lib/kubelet/pods/1797f7cf-e20a-4121-ba36-a22cb46feff2/volumes" Jul 09 14:07:13.886405 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.886367 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:07:13.886836 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.886764 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1797f7cf-e20a-4121-ba36-a22cb46feff2" containerName="console" Jul 09 14:07:13.886836 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.886777 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1797f7cf-e20a-4121-ba36-a22cb46feff2" containerName="console" Jul 09 14:07:13.886905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.886844 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1797f7cf-e20a-4121-ba36-a22cb46feff2" containerName="console" Jul 09 14:07:13.889795 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.889770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.901480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.901453 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:07:13.962206 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962165 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962306 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962331 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824bv\" (UniqueName: \"kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:13.962404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:13.962380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063633 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824bv\" (UniqueName: \"kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063753 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.063824 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.063781 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.064499 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.064472 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.064624 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.064542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.064624 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.064542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.064788 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.064763 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.066127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.066100 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.066361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.066339 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.072675 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.072653 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824bv\" (UniqueName: \"kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv\") pod \"console-658cccdd8b-qwcvk\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.200749 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.200661 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:14.339404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:14.339379 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:07:14.344219 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:07:14.344189 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20315ab9_aab2_4669_bbb1_66954b679fb4.slice/crio-511ab5fce7842e183f388667fd44983226dbbb742ad57ee138678d4be3ccf266 WatchSource:0}: Error finding container 511ab5fce7842e183f388667fd44983226dbbb742ad57ee138678d4be3ccf266: Status 404 returned error can't find the container with id 511ab5fce7842e183f388667fd44983226dbbb742ad57ee138678d4be3ccf266 Jul 09 14:07:15.226441 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:15.226394 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658cccdd8b-qwcvk" event={"ID":"20315ab9-aab2-4669-bbb1-66954b679fb4","Type":"ContainerStarted","Data":"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c"} Jul 09 14:07:15.226868 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:15.226458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658cccdd8b-qwcvk" event={"ID":"20315ab9-aab2-4669-bbb1-66954b679fb4","Type":"ContainerStarted","Data":"511ab5fce7842e183f388667fd44983226dbbb742ad57ee138678d4be3ccf266"} Jul 09 14:07:15.246155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:15.246098 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658cccdd8b-qwcvk" podStartSLOduration=2.246073328 podStartE2EDuration="2.246073328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:15.245041157 +0000 UTC m=+296.883715944" watchObservedRunningTime="2026-07-09 14:07:15.246073328 +0000 UTC m=+296.884748105" Jul 09 14:07:18.796755 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:18.796724 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:07:18.800810 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:18.800786 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:07:18.801918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:18.801899 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:07:18.805975 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:18.805957 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:07:18.807911 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:18.807893 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 14:07:24.200883 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:24.200842 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:24.203579 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:24.200945 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:24.205761 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:24.205736 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:24.271760 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:24.271732 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:07:24.315020 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:24.314982 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:07:40.212215 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.212179 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq"] Jul 09 14:07:40.216138 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.216111 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.219019 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.218992 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:07:40.220095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.220078 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j798r\"" Jul 09 14:07:40.220206 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.220080 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:07:40.226737 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.226707 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq"] Jul 09 14:07:40.296808 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.296764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.297012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.296819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9q9\" (UniqueName: \"kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.297012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.296953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.397821 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.397762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.398029 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.397834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9q9\" (UniqueName: \"kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.398029 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.397903 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.398201 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.398175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.398319 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.398298 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.406818 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.406786 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9q9\" (UniqueName: \"kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.527869 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.527772 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:07:40.671714 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.671652 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq"] Jul 09 14:07:40.674023 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:07:40.673985 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4e6ed9_1933_402f_9034_1fe062b672ea.slice/crio-3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b WatchSource:0}: Error finding container 3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b: Status 404 returned error can't find the container with id 3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b Jul 09 14:07:40.675987 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:40.675969 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:07:41.348991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:41.348949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" event={"ID":"7c4e6ed9-1933-402f-9034-1fe062b672ea","Type":"ContainerStarted","Data":"3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b"} Jul 09 14:07:49.339794 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.339744 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-846b67c96f-z5ldg" podUID="a0ba67ae-7728-403d-902d-f0d273331803" containerName="console" containerID="cri-o://99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08" gracePeriod=15 Jul 09 14:07:49.387903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.387864 2576 generic.go:356] "Generic (PLEG): container finished" podID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerID="3244cc2091b324bb2fba2fd3eb03b6ed5eedd25950f23da5c5b9bb236cfd9dc5" exitCode=0 Jul 09 14:07:49.388075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.387948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" event={"ID":"7c4e6ed9-1933-402f-9034-1fe062b672ea","Type":"ContainerDied","Data":"3244cc2091b324bb2fba2fd3eb03b6ed5eedd25950f23da5c5b9bb236cfd9dc5"} Jul 09 14:07:49.585173 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.585146 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:07:49.694738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694651 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.694738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694685 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.694738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694706 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.695011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694757 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.695011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694777 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.695011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694814 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.695011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.694859 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrmqz\" (UniqueName: \"kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz\") pod \"a0ba67ae-7728-403d-902d-f0d273331803\" (UID: \"a0ba67ae-7728-403d-902d-f0d273331803\") " Jul 09 14:07:49.695208 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.695141 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config" (OuterVolumeSpecName: "console-config") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:49.695208 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.695179 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca" (OuterVolumeSpecName: "service-ca") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:49.695331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.695254 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:49.695331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.695268 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:49.696911 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.696880 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:07:49.696911 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.696901 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:07:49.697050 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.696928 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz" (OuterVolumeSpecName: "kube-api-access-wrmqz") pod "a0ba67ae-7728-403d-902d-f0d273331803" (UID: "a0ba67ae-7728-403d-902d-f0d273331803"). InnerVolumeSpecName "kube-api-access-wrmqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:49.795859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795818 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrmqz\" (UniqueName: \"kubernetes.io/projected/a0ba67ae-7728-403d-902d-f0d273331803-kube-api-access-wrmqz\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.795859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795851 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-oauth-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.795859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795862 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-console-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.796092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795871 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-oauth-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.796092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795879 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-trusted-ca-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.796092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795889 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba67ae-7728-403d-902d-f0d273331803-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:49.796092 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:49.795898 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0ba67ae-7728-403d-902d-f0d273331803-console-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:07:50.394496 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.394455 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0ba67ae-7728-403d-902d-f0d273331803" containerID="99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08" exitCode=0 Jul 09 14:07:50.394971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.394503 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846b67c96f-z5ldg" Jul 09 14:07:50.394971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.394511 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846b67c96f-z5ldg" event={"ID":"a0ba67ae-7728-403d-902d-f0d273331803","Type":"ContainerDied","Data":"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08"} Jul 09 14:07:50.394971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.394559 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846b67c96f-z5ldg" event={"ID":"a0ba67ae-7728-403d-902d-f0d273331803","Type":"ContainerDied","Data":"e1a79427f1607cff32e8188cde2ab521d45ce2fb0b7f13980cefac666d180c1e"} Jul 09 14:07:50.394971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.394587 2576 scope.go:117] "RemoveContainer" containerID="99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08" Jul 09 14:07:50.405785 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.405761 2576 scope.go:117] "RemoveContainer" containerID="99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08" Jul 09 14:07:50.406100 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:07:50.406074 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08\": container with ID starting with 99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08 not found: ID does not exist" containerID="99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08" Jul 09 14:07:50.406159 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.406113 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08"} err="failed to get container status \"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08\": rpc error: code = NotFound desc = could not find container \"99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08\": container with ID starting with 99f3b4a0f11ea3d91ea1fe48a1371b48b341ee3bec0fd34f6e090b9ff6d3fd08 not found: ID does not exist" Jul 09 14:07:50.420179 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.420136 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:07:50.424361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.424334 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-846b67c96f-z5ldg"] Jul 09 14:07:50.894947 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:50.894907 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ba67ae-7728-403d-902d-f0d273331803" path="/var/lib/kubelet/pods/a0ba67ae-7728-403d-902d-f0d273331803/volumes" Jul 09 14:07:52.408331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:52.408285 2576 generic.go:356] "Generic (PLEG): container finished" podID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerID="5826efeb849d492715d863038af05b75789c03902d921a3215cac64340d727b6" exitCode=0 Jul 09 14:07:52.408701 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:07:52.408375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" event={"ID":"7c4e6ed9-1933-402f-9034-1fe062b672ea","Type":"ContainerDied","Data":"5826efeb849d492715d863038af05b75789c03902d921a3215cac64340d727b6"} Jul 09 14:08:02.460852 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:02.460817 2576 generic.go:356] "Generic (PLEG): container finished" podID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerID="ed12871d9c69a5431ba8d59e25a9880304989fc0b537af79dd790a7b40c4a3f4" exitCode=0 Jul 09 14:08:02.461280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:02.460904 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" event={"ID":"7c4e6ed9-1933-402f-9034-1fe062b672ea","Type":"ContainerDied","Data":"ed12871d9c69a5431ba8d59e25a9880304989fc0b537af79dd790a7b40c4a3f4"} Jul 09 14:08:03.608074 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.608037 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:08:03.729924 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.729825 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util\") pod \"7c4e6ed9-1933-402f-9034-1fe062b672ea\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " Jul 09 14:08:03.729924 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.729901 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k9q9\" (UniqueName: \"kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9\") pod \"7c4e6ed9-1933-402f-9034-1fe062b672ea\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " Jul 09 14:08:03.730144 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.729968 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle\") pod \"7c4e6ed9-1933-402f-9034-1fe062b672ea\" (UID: \"7c4e6ed9-1933-402f-9034-1fe062b672ea\") " Jul 09 14:08:03.730660 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.730624 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle" (OuterVolumeSpecName: "bundle") pod "7c4e6ed9-1933-402f-9034-1fe062b672ea" (UID: "7c4e6ed9-1933-402f-9034-1fe062b672ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:03.732097 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.732071 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9" (OuterVolumeSpecName: "kube-api-access-9k9q9") pod "7c4e6ed9-1933-402f-9034-1fe062b672ea" (UID: "7c4e6ed9-1933-402f-9034-1fe062b672ea"). InnerVolumeSpecName "kube-api-access-9k9q9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:08:03.737290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.737259 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util" (OuterVolumeSpecName: "util") pod "7c4e6ed9-1933-402f-9034-1fe062b672ea" (UID: "7c4e6ed9-1933-402f-9034-1fe062b672ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:03.831410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.831377 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:03.831410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.831404 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c4e6ed9-1933-402f-9034-1fe062b672ea-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:03.831410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:03.831414 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k9q9\" (UniqueName: \"kubernetes.io/projected/7c4e6ed9-1933-402f-9034-1fe062b672ea-kube-api-access-9k9q9\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:04.471566 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:04.471536 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" Jul 09 14:08:04.471566 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:04.471549 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k76vq" event={"ID":"7c4e6ed9-1933-402f-9034-1fe062b672ea","Type":"ContainerDied","Data":"3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b"} Jul 09 14:08:04.471761 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:04.471589 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d24136b45a955e2e718fce1e496bada94c7ef9fd3c5acb11e7fc05ec6594b2b" Jul 09 14:08:08.277327 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277288 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r"] Jul 09 14:08:08.277707 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277669 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="extract" Jul 09 14:08:08.277707 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277683 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="extract" Jul 09 14:08:08.277707 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277698 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="util" Jul 09 14:08:08.277707 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277705 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="util" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277722 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ba67ae-7728-403d-902d-f0d273331803" containerName="console" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277727 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ba67ae-7728-403d-902d-f0d273331803" containerName="console" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277735 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="pull" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277740 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="pull" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277789 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ba67ae-7728-403d-902d-f0d273331803" containerName="console" Jul 09 14:08:08.277861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.277798 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4e6ed9-1933-402f-9034-1fe062b672ea" containerName="extract" Jul 09 14:08:08.279841 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.279824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.282480 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.282458 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:08:08.282669 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.282649 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lzngn\"" Jul 09 14:08:08.282766 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.282655 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 14:08:08.290864 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.290838 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r"] Jul 09 14:08:08.375356 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.375319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99dk6\" (UniqueName: \"kubernetes.io/projected/204cf422-35f2-4c7d-9b30-232e08da67b2-kube-api-access-99dk6\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.375541 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.375443 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204cf422-35f2-4c7d-9b30-232e08da67b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.476601 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.476564 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204cf422-35f2-4c7d-9b30-232e08da67b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.476714 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.476648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99dk6\" (UniqueName: \"kubernetes.io/projected/204cf422-35f2-4c7d-9b30-232e08da67b2-kube-api-access-99dk6\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.476972 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.476953 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204cf422-35f2-4c7d-9b30-232e08da67b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.485460 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.485427 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99dk6\" (UniqueName: \"kubernetes.io/projected/204cf422-35f2-4c7d-9b30-232e08da67b2-kube-api-access-99dk6\") pod \"cert-manager-operator-controller-manager-55c444bf86-qs99r\" (UID: \"204cf422-35f2-4c7d-9b30-232e08da67b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.589239 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.589200 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" Jul 09 14:08:08.724029 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:08.723940 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r"] Jul 09 14:08:08.726705 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:08:08.726670 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204cf422_35f2_4c7d_9b30_232e08da67b2.slice/crio-6374e34012884160a0f7ebf35dd889c533dcad8a122e5caa2060c55b481346ed WatchSource:0}: Error finding container 6374e34012884160a0f7ebf35dd889c533dcad8a122e5caa2060c55b481346ed: Status 404 returned error can't find the container with id 6374e34012884160a0f7ebf35dd889c533dcad8a122e5caa2060c55b481346ed Jul 09 14:08:09.495693 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:09.495652 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" event={"ID":"204cf422-35f2-4c7d-9b30-232e08da67b2","Type":"ContainerStarted","Data":"6374e34012884160a0f7ebf35dd889c533dcad8a122e5caa2060c55b481346ed"} Jul 09 14:08:25.575806 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:25.575763 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" event={"ID":"204cf422-35f2-4c7d-9b30-232e08da67b2","Type":"ContainerStarted","Data":"acef87bd9c26deda976ca0bcdb1ff13068e3a403ae775497747c18d373e2e4f6"} Jul 09 14:08:25.599691 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:25.599420 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qs99r" podStartSLOduration=1.000349241 podStartE2EDuration="17.599398788s" totalImagesPullingTime="16.599049547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:25.597197021 +0000 UTC m=+367.235871826" watchObservedRunningTime="2026-07-09 14:08:25.599398788 +0000 UTC m=+367.238073563" Jul 09 14:08:26.249938 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.249898 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z"] Jul 09 14:08:26.254018 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.253990 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.256940 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.256921 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j798r\"" Jul 09 14:08:26.257401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.257375 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:08:26.258196 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.258174 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:08:26.278626 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.278599 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z"] Jul 09 14:08:26.337744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.337708 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xjpx\" (UniqueName: \"kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.337913 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.337770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.337913 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.337813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.438814 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.438773 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.438981 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.438832 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.438981 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.438923 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjpx\" (UniqueName: \"kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.439150 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.439128 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.439214 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.439192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.450044 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.450017 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjpx\" (UniqueName: \"kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.563852 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.563822 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:26.706882 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:26.706855 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z"] Jul 09 14:08:26.708938 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:08:26.708913 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca05244_6e2f_431d_a5bd_f19d94526df0.slice/crio-641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c WatchSource:0}: Error finding container 641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c: Status 404 returned error can't find the container with id 641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c Jul 09 14:08:27.587317 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:27.587271 2576 generic.go:356] "Generic (PLEG): container finished" podID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerID="0e89a53fed294a0d37ecf614e2b99125403958b51a0dad0a91b2a4e7458478da" exitCode=0 Jul 09 14:08:27.587510 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:27.587373 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" event={"ID":"dca05244-6e2f-431d-a5bd-f19d94526df0","Type":"ContainerDied","Data":"0e89a53fed294a0d37ecf614e2b99125403958b51a0dad0a91b2a4e7458478da"} Jul 09 14:08:27.587510 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:27.587432 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" event={"ID":"dca05244-6e2f-431d-a5bd-f19d94526df0","Type":"ContainerStarted","Data":"641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c"} Jul 09 14:08:28.967745 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.967717 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm"] Jul 09 14:08:28.970906 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.970884 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:28.973991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.973966 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 14:08:28.975114 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.975096 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 14:08:28.975114 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.975102 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-r8ptm\"" Jul 09 14:08:28.981409 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:28.981386 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm"] Jul 09 14:08:29.063880 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.063839 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmd2l\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-kube-api-access-kmd2l\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.064055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.063970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.165366 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.165326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.165563 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.165390 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmd2l\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-kube-api-access-kmd2l\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.173605 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.173578 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.173730 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.173646 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmd2l\" (UniqueName: \"kubernetes.io/projected/925f241d-04ef-4470-8365-95e20ea00c2f-kube-api-access-kmd2l\") pod \"cert-manager-cainjector-559cdd8c87-z2vzm\" (UID: \"925f241d-04ef-4470-8365-95e20ea00c2f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.290425 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.290336 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" Jul 09 14:08:29.426029 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.425981 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm"] Jul 09 14:08:29.429892 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:08:29.429850 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod925f241d_04ef_4470_8365_95e20ea00c2f.slice/crio-eb2eb11a9d647e7f18387c8322a1d4565c71504e2d1d65376b843c056b3e8dcf WatchSource:0}: Error finding container eb2eb11a9d647e7f18387c8322a1d4565c71504e2d1d65376b843c056b3e8dcf: Status 404 returned error can't find the container with id eb2eb11a9d647e7f18387c8322a1d4565c71504e2d1d65376b843c056b3e8dcf Jul 09 14:08:29.598586 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:29.598546 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" event={"ID":"925f241d-04ef-4470-8365-95e20ea00c2f","Type":"ContainerStarted","Data":"eb2eb11a9d647e7f18387c8322a1d4565c71504e2d1d65376b843c056b3e8dcf"} Jul 09 14:08:31.611444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:31.611404 2576 generic.go:356] "Generic (PLEG): container finished" podID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerID="9fde521a0cc718da70aad9b03349b2c16b6af608c5b8d60107a21e06f8a888aa" exitCode=0 Jul 09 14:08:31.611965 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:31.611498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" event={"ID":"dca05244-6e2f-431d-a5bd-f19d94526df0","Type":"ContainerDied","Data":"9fde521a0cc718da70aad9b03349b2c16b6af608c5b8d60107a21e06f8a888aa"} Jul 09 14:08:32.620472 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:32.620433 2576 generic.go:356] "Generic (PLEG): container finished" podID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerID="a08acaba820eacce437f5862c6438085bb77d3d53800640100c0a0c20613d127" exitCode=0 Jul 09 14:08:32.620871 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:32.620488 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" event={"ID":"dca05244-6e2f-431d-a5bd-f19d94526df0","Type":"ContainerDied","Data":"a08acaba820eacce437f5862c6438085bb77d3d53800640100c0a0c20613d127"} Jul 09 14:08:33.629353 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.629256 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" event={"ID":"925f241d-04ef-4470-8365-95e20ea00c2f","Type":"ContainerStarted","Data":"f0f51076cdba7e480a7357cfe723157c5dd91addfe24636b3739fea02447af27"} Jul 09 14:08:33.646645 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.646574 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z2vzm" podStartSLOduration=1.711957603 podStartE2EDuration="5.646554751s" totalImagesPullingTime="3.934597148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:33.644890278 +0000 UTC m=+375.283565063" watchObservedRunningTime="2026-07-09 14:08:33.646554751 +0000 UTC m=+375.285229525" Jul 09 14:08:33.788583 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.788555 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:33.914424 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.914341 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle\") pod \"dca05244-6e2f-431d-a5bd-f19d94526df0\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " Jul 09 14:08:33.914424 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.914379 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xjpx\" (UniqueName: \"kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx\") pod \"dca05244-6e2f-431d-a5bd-f19d94526df0\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " Jul 09 14:08:33.914665 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.914427 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util\") pod \"dca05244-6e2f-431d-a5bd-f19d94526df0\" (UID: \"dca05244-6e2f-431d-a5bd-f19d94526df0\") " Jul 09 14:08:33.914723 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.914674 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle" (OuterVolumeSpecName: "bundle") pod "dca05244-6e2f-431d-a5bd-f19d94526df0" (UID: "dca05244-6e2f-431d-a5bd-f19d94526df0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:33.916478 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.916455 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx" (OuterVolumeSpecName: "kube-api-access-6xjpx") pod "dca05244-6e2f-431d-a5bd-f19d94526df0" (UID: "dca05244-6e2f-431d-a5bd-f19d94526df0"). InnerVolumeSpecName "kube-api-access-6xjpx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:08:33.921882 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:33.921855 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util" (OuterVolumeSpecName: "util") pod "dca05244-6e2f-431d-a5bd-f19d94526df0" (UID: "dca05244-6e2f-431d-a5bd-f19d94526df0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:34.015411 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.015372 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:34.015411 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.015404 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xjpx\" (UniqueName: \"kubernetes.io/projected/dca05244-6e2f-431d-a5bd-f19d94526df0-kube-api-access-6xjpx\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:34.015411 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.015418 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca05244-6e2f-431d-a5bd-f19d94526df0-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:34.635898 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.635867 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" Jul 09 14:08:34.636328 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.635895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4pg7z" event={"ID":"dca05244-6e2f-431d-a5bd-f19d94526df0","Type":"ContainerDied","Data":"641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c"} Jul 09 14:08:34.636328 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:34.635941 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="641a45c2fa02e3b6f5ae8d9fa2d76f8fdcee52ecba6d33c8f802379b9a29317c" Jul 09 14:08:37.005191 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005152 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-898f2"] Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005518 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="pull" Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005530 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="pull" Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005547 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="extract" Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005552 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="extract" Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005565 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="util" Jul 09 14:08:37.005589 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005572 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="util" Jul 09 14:08:37.005792 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.005625 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca05244-6e2f-431d-a5bd-f19d94526df0" containerName="extract" Jul 09 14:08:37.007708 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.007688 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.010264 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.010241 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hcbd8\"" Jul 09 14:08:37.017673 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.017645 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-898f2"] Jul 09 14:08:37.143483 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.143442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-bound-sa-token\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.143687 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.143590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrx7b\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-kube-api-access-jrx7b\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.244148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.244109 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-bound-sa-token\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.244384 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.244257 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrx7b\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-kube-api-access-jrx7b\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.253410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.253384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-bound-sa-token\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.253525 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.253441 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrx7b\" (UniqueName: \"kubernetes.io/projected/5ee0323e-4aa7-4730-8874-27dff68dedea-kube-api-access-jrx7b\") pod \"cert-manager-55b4d954d5-898f2\" (UID: \"5ee0323e-4aa7-4730-8874-27dff68dedea\") " pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.317059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.317025 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-898f2" Jul 09 14:08:37.454717 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.454690 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-898f2"] Jul 09 14:08:37.457240 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:08:37.457207 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ee0323e_4aa7_4730_8874_27dff68dedea.slice/crio-039a6c061139eaaff8cf0d8fcea4429c99bc73b85b9fe06482dfdb0b43f61627 WatchSource:0}: Error finding container 039a6c061139eaaff8cf0d8fcea4429c99bc73b85b9fe06482dfdb0b43f61627: Status 404 returned error can't find the container with id 039a6c061139eaaff8cf0d8fcea4429c99bc73b85b9fe06482dfdb0b43f61627 Jul 09 14:08:37.651617 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.651516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-898f2" event={"ID":"5ee0323e-4aa7-4730-8874-27dff68dedea","Type":"ContainerStarted","Data":"f111de6954c19632c189f709fe5fcf9f27abb5c396287ae2eccd1fba8331b673"} Jul 09 14:08:37.651617 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.651562 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-898f2" event={"ID":"5ee0323e-4aa7-4730-8874-27dff68dedea","Type":"ContainerStarted","Data":"039a6c061139eaaff8cf0d8fcea4429c99bc73b85b9fe06482dfdb0b43f61627"} Jul 09 14:08:37.670398 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:37.670176 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-898f2" podStartSLOduration=1.67013647 podStartE2EDuration="1.67013647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:37.669936421 +0000 UTC m=+379.308611197" watchObservedRunningTime="2026-07-09 14:08:37.67013647 +0000 UTC m=+379.308811262" Jul 09 14:08:46.625091 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.625050 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g"] Jul 09 14:08:46.628747 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.628727 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.631460 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.631432 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:08:46.631460 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.631434 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:08:46.632676 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.632658 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j798r\"" Jul 09 14:08:46.637687 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.637663 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g"] Jul 09 14:08:46.724591 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.724553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.724744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.724654 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm7ht\" (UniqueName: \"kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.724744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.724707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.825540 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.825485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm7ht\" (UniqueName: \"kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.825540 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.825554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.825751 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.825606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.826041 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.826020 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.826084 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.826037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.833830 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.833798 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm7ht\" (UniqueName: \"kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:46.939407 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:46.939319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:47.075795 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:47.075770 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g"] Jul 09 14:08:47.077873 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:08:47.077844 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb870d011_ae8f_4c25_99cd_4034569b1231.slice/crio-ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e WatchSource:0}: Error finding container ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e: Status 404 returned error can't find the container with id ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e Jul 09 14:08:47.702112 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:47.702078 2576 generic.go:356] "Generic (PLEG): container finished" podID="b870d011-ae8f-4c25-99cd-4034569b1231" containerID="72057bb047feaf66a7f3deb8d47a6182dabdeb77a0915a58404de1b9ae76863e" exitCode=0 Jul 09 14:08:47.702555 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:47.702167 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" event={"ID":"b870d011-ae8f-4c25-99cd-4034569b1231","Type":"ContainerDied","Data":"72057bb047feaf66a7f3deb8d47a6182dabdeb77a0915a58404de1b9ae76863e"} Jul 09 14:08:47.702555 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:47.702213 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" event={"ID":"b870d011-ae8f-4c25-99cd-4034569b1231","Type":"ContainerStarted","Data":"ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e"} Jul 09 14:08:52.727410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:52.727375 2576 generic.go:356] "Generic (PLEG): container finished" podID="b870d011-ae8f-4c25-99cd-4034569b1231" containerID="45924830b026de1580284825b55b3d1af61175ee10e107878b8caf5389715782" exitCode=0 Jul 09 14:08:52.727810 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:52.727465 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" event={"ID":"b870d011-ae8f-4c25-99cd-4034569b1231","Type":"ContainerDied","Data":"45924830b026de1580284825b55b3d1af61175ee10e107878b8caf5389715782"} Jul 09 14:08:53.734713 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:53.734677 2576 generic.go:356] "Generic (PLEG): container finished" podID="b870d011-ae8f-4c25-99cd-4034569b1231" containerID="ddcff18f2c31d4733da953213661dd0c6d11e3682f4f3e646c471a3356e70ab0" exitCode=0 Jul 09 14:08:53.735119 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:53.734765 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" event={"ID":"b870d011-ae8f-4c25-99cd-4034569b1231","Type":"ContainerDied","Data":"ddcff18f2c31d4733da953213661dd0c6d11e3682f4f3e646c471a3356e70ab0"} Jul 09 14:08:54.871664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:54.871637 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:54.999337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:54.999214 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm7ht\" (UniqueName: \"kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht\") pod \"b870d011-ae8f-4c25-99cd-4034569b1231\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " Jul 09 14:08:54.999337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:54.999286 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle\") pod \"b870d011-ae8f-4c25-99cd-4034569b1231\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " Jul 09 14:08:54.999337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:54.999322 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util\") pod \"b870d011-ae8f-4c25-99cd-4034569b1231\" (UID: \"b870d011-ae8f-4c25-99cd-4034569b1231\") " Jul 09 14:08:55.000011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:54.999981 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle" (OuterVolumeSpecName: "bundle") pod "b870d011-ae8f-4c25-99cd-4034569b1231" (UID: "b870d011-ae8f-4c25-99cd-4034569b1231"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:55.001568 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.001526 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht" (OuterVolumeSpecName: "kube-api-access-pm7ht") pod "b870d011-ae8f-4c25-99cd-4034569b1231" (UID: "b870d011-ae8f-4c25-99cd-4034569b1231"). InnerVolumeSpecName "kube-api-access-pm7ht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:08:55.004681 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.004656 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util" (OuterVolumeSpecName: "util") pod "b870d011-ae8f-4c25-99cd-4034569b1231" (UID: "b870d011-ae8f-4c25-99cd-4034569b1231"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:08:55.100224 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.100180 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm7ht\" (UniqueName: \"kubernetes.io/projected/b870d011-ae8f-4c25-99cd-4034569b1231-kube-api-access-pm7ht\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:55.100224 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.100217 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:55.100224 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.100226 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b870d011-ae8f-4c25-99cd-4034569b1231-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:08:55.746221 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.746172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" event={"ID":"b870d011-ae8f-4c25-99cd-4034569b1231","Type":"ContainerDied","Data":"ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e"} Jul 09 14:08:55.746221 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.746208 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcrp8g" Jul 09 14:08:55.746454 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:08:55.746266 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff1ae3b28a89c2d6f915391231210d8553681dad957e0478596b196388f21c0e" Jul 09 14:09:05.974307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.974258 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp"] Jul 09 14:09:05.974938 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.974919 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="util" Jul 09 14:09:05.974982 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.974943 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="util" Jul 09 14:09:05.974982 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.974971 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="pull" Jul 09 14:09:05.975040 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.974980 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="pull" Jul 09 14:09:05.975040 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.975007 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="extract" Jul 09 14:09:05.975040 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.975019 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="extract" Jul 09 14:09:05.975193 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.975179 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b870d011-ae8f-4c25-99cd-4034569b1231" containerName="extract" Jul 09 14:09:05.980559 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.980531 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:05.983521 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.983491 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qwmbp\"" Jul 09 14:09:05.983662 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.983526 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 14:09:05.983875 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.983857 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 14:09:05.983920 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.983910 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 14:09:05.984001 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.983857 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 14:09:05.993612 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:05.993574 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp"] Jul 09 14:09:06.098827 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.098782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.099047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.098834 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-tmp\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.099047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.098917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.099047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.099013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-manifests\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.099182 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.099091 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw4fw\" (UniqueName: \"kubernetes.io/projected/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-kube-api-access-sw4fw\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.200552 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.200507 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw4fw\" (UniqueName: \"kubernetes.io/projected/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-kube-api-access-sw4fw\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.200762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.200561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.200762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.200594 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-tmp\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.200762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.200635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.200762 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.200693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-manifests\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.201109 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.201085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-manifests\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.202946 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.202914 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-tmp\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.203069 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.203050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.203148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.203131 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.211769 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.211739 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw4fw\" (UniqueName: \"kubernetes.io/projected/b7af6e15-9a7d-4ba3-868a-d64570b3ffff-kube-api-access-sw4fw\") pod \"opendatahub-operator-controller-manager-6f67568454-wtfsp\" (UID: \"b7af6e15-9a7d-4ba3-868a-d64570b3ffff\") " pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.294404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.294318 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:06.447175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.447149 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp"] Jul 09 14:09:06.449498 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:06.449464 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7af6e15_9a7d_4ba3_868a_d64570b3ffff.slice/crio-bd9dd0906d598a4d6986e96e7e706a82a0bea821943bc19f9db64cd4671d1304 WatchSource:0}: Error finding container bd9dd0906d598a4d6986e96e7e706a82a0bea821943bc19f9db64cd4671d1304: Status 404 returned error can't find the container with id bd9dd0906d598a4d6986e96e7e706a82a0bea821943bc19f9db64cd4671d1304 Jul 09 14:09:06.806869 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:06.806834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" event={"ID":"b7af6e15-9a7d-4ba3-868a-d64570b3ffff","Type":"ContainerStarted","Data":"bd9dd0906d598a4d6986e96e7e706a82a0bea821943bc19f9db64cd4671d1304"} Jul 09 14:09:09.827401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:09.827368 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" event={"ID":"b7af6e15-9a7d-4ba3-868a-d64570b3ffff","Type":"ContainerStarted","Data":"c74442ead6821efcec922a84dc026b7f083b3e2676ea65236a80f1e2bc318354"} Jul 09 14:09:10.834386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:10.834294 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7af6e15-9a7d-4ba3-868a-d64570b3ffff" containerID="c74442ead6821efcec922a84dc026b7f083b3e2676ea65236a80f1e2bc318354" exitCode=0 Jul 09 14:09:10.834750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:10.834380 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" event={"ID":"b7af6e15-9a7d-4ba3-868a-d64570b3ffff","Type":"ContainerDied","Data":"c74442ead6821efcec922a84dc026b7f083b3e2676ea65236a80f1e2bc318354"} Jul 09 14:09:11.841017 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:11.840978 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" event={"ID":"b7af6e15-9a7d-4ba3-868a-d64570b3ffff","Type":"ContainerStarted","Data":"f25e87ab2bef052eba381accceb537bcee5a4efef231644d9017fd0b80ca51da"} Jul 09 14:09:11.841410 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:11.841080 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:11.866993 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:11.866943 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" podStartSLOduration=2.430781925 podStartE2EDuration="6.866927398s" totalImagesPullingTime="3.384070819s" totalInitContainerRuntime="1.052074654s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:11.863974289 +0000 UTC m=+413.502649117" watchObservedRunningTime="2026-07-09 14:09:11.866927398 +0000 UTC m=+413.505602170" Jul 09 14:09:22.848319 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:22.848278 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f67568454-wtfsp" Jul 09 14:09:27.511639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.511597 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf"] Jul 09 14:09:27.520968 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.520935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.524938 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.524909 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 14:09:27.525095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.524910 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 14:09:27.525095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.524910 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pvbcl\"" Jul 09 14:09:27.538201 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.538171 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf"] Jul 09 14:09:27.599628 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.599596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tmp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.599816 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.599639 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhmkp\" (UniqueName: \"kubernetes.io/projected/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-kube-api-access-xhmkp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.599816 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.599772 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.700772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.700734 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tmp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.700772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.700775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhmkp\" (UniqueName: \"kubernetes.io/projected/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-kube-api-access-xhmkp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.701016 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.700851 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.701016 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:09:27.700964 2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 09 14:09:27.701182 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:09:27.701029 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs podName:7eb16642-3fdf-4ffe-bdab-76d0510a93ef nodeName:}" failed. No retries permitted until 2026-07-09 14:09:28.201010255 +0000 UTC m=+429.839685024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs") pod "kube-auth-proxy-79b6dccbd4-wkmbf" (UID: "7eb16642-3fdf-4ffe-bdab-76d0510a93ef") : secret "kube-auth-proxy-tls" not found Jul 09 14:09:27.703075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.703043 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tmp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.726006 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.725966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhmkp\" (UniqueName: \"kubernetes.io/projected/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-kube-api-access-xhmkp\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:27.734343 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.734311 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tfxx6"] Jul 09 14:09:27.738259 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.738215 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.740841 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.740815 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 14:09:27.740841 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.740835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wzn9h\"" Jul 09 14:09:27.753969 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.753943 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tfxx6"] Jul 09 14:09:27.802074 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.801989 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c70fa9f-1b94-4e60-aec5-3045950aa46d-cert\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.802074 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.802046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxtnt\" (UniqueName: \"kubernetes.io/projected/3c70fa9f-1b94-4e60-aec5-3045950aa46d-kube-api-access-lxtnt\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.902896 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.902855 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c70fa9f-1b94-4e60-aec5-3045950aa46d-cert\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.903068 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.902918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxtnt\" (UniqueName: \"kubernetes.io/projected/3c70fa9f-1b94-4e60-aec5-3045950aa46d-kube-api-access-lxtnt\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.905371 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.905346 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c70fa9f-1b94-4e60-aec5-3045950aa46d-cert\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:27.929864 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:27.929823 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxtnt\" (UniqueName: \"kubernetes.io/projected/3c70fa9f-1b94-4e60-aec5-3045950aa46d-kube-api-access-lxtnt\") pod \"odh-model-controller-7998c67dc4-tfxx6\" (UID: \"3c70fa9f-1b94-4e60-aec5-3045950aa46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:28.049372 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.049338 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:28.206635 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.205788 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tfxx6"] Jul 09 14:09:28.207923 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.207450 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:28.211197 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.210716 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb16642-3fdf-4ffe-bdab-76d0510a93ef-tls-certs\") pod \"kube-auth-proxy-79b6dccbd4-wkmbf\" (UID: \"7eb16642-3fdf-4ffe-bdab-76d0510a93ef\") " pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:28.212943 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:28.212921 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c70fa9f_1b94_4e60_aec5_3045950aa46d.slice/crio-47ea3d6f9b732372df18e7f25d67f033552274abf124ab88fd4e5f38dc5e7fbf WatchSource:0}: Error finding container 47ea3d6f9b732372df18e7f25d67f033552274abf124ab88fd4e5f38dc5e7fbf: Status 404 returned error can't find the container with id 47ea3d6f9b732372df18e7f25d67f033552274abf124ab88fd4e5f38dc5e7fbf Jul 09 14:09:28.431710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.431678 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" Jul 09 14:09:28.472335 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.472287 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cndlx"] Jul 09 14:09:28.477166 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.477135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.481013 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.480979 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 14:09:28.495309 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.495278 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cndlx"] Jul 09 14:09:28.596646 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.596614 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf"] Jul 09 14:09:28.598496 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:28.598467 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb16642_3fdf_4ffe_bdab_76d0510a93ef.slice/crio-fa1861a5903735315afb5ef85eefe24b2e1dbc1a054fe9eb10eabd11916f5e4f WatchSource:0}: Error finding container fa1861a5903735315afb5ef85eefe24b2e1dbc1a054fe9eb10eabd11916f5e4f: Status 404 returned error can't find the container with id fa1861a5903735315afb5ef85eefe24b2e1dbc1a054fe9eb10eabd11916f5e4f Jul 09 14:09:28.611790 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.611763 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.611893 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.611812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhkrz\" (UniqueName: \"kubernetes.io/projected/4cfae5e3-218f-41e7-9c62-258cabbb800e-kube-api-access-mhkrz\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.713016 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.712915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.713016 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.712987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhkrz\" (UniqueName: \"kubernetes.io/projected/4cfae5e3-218f-41e7-9c62-258cabbb800e-kube-api-access-mhkrz\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.713559 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:09:28.713521 2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 14:09:28.713779 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:09:28.713761 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs podName:4cfae5e3-218f-41e7-9c62-258cabbb800e nodeName:}" failed. No retries permitted until 2026-07-09 14:09:29.213737695 +0000 UTC m=+430.852412456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs") pod "model-serving-api-54d89bd5fc-cndlx" (UID: "4cfae5e3-218f-41e7-9c62-258cabbb800e") : secret "model-serving-api-tls" not found Jul 09 14:09:28.722976 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.722942 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhkrz\" (UniqueName: \"kubernetes.io/projected/4cfae5e3-218f-41e7-9c62-258cabbb800e-kube-api-access-mhkrz\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:28.934910 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.934869 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" event={"ID":"7eb16642-3fdf-4ffe-bdab-76d0510a93ef","Type":"ContainerStarted","Data":"fa1861a5903735315afb5ef85eefe24b2e1dbc1a054fe9eb10eabd11916f5e4f"} Jul 09 14:09:28.936099 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.936066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" event={"ID":"3c70fa9f-1b94-4e60-aec5-3045950aa46d","Type":"ContainerStarted","Data":"47ea3d6f9b732372df18e7f25d67f033552274abf124ab88fd4e5f38dc5e7fbf"} Jul 09 14:09:28.949012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.948979 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:28.954019 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.953989 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:28.956670 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.956649 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 14:09:28.956820 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.956772 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 14:09:28.956926 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.956900 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 14:09:28.957111 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.957097 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 14:09:28.957166 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.957120 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vpkhx\"" Jul 09 14:09:28.968726 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:28.968665 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:29.016167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89bv\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016340 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.016485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.016432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.117711 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.117661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.117891 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.117722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.117891 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.117749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.117891 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.117856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d89bv\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.118043 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.117966 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.118043 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.118028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.118141 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.118056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.118436 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.118404 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.120786 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.120758 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.120918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.120837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.120918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.120896 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.121055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.121036 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.129192 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.129162 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.129391 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.129293 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89bv\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv\") pod \"istiod-openshift-gateway-7574459656-bhpkv\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.218873 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.218771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:29.221584 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.221558 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfae5e3-218f-41e7-9c62-258cabbb800e-tls-certs\") pod \"model-serving-api-54d89bd5fc-cndlx\" (UID: \"4cfae5e3-218f-41e7-9c62-258cabbb800e\") " pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:29.264293 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.264256 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:29.395834 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.395515 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:29.433047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.433009 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:29.439456 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:29.439409 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fa2904_5d6d_4615_b349_46793ed579eb.slice/crio-4221aaf723213d7e7fdd5081574ae1e4de2356c46fb5c4a067cd8823b93fc660 WatchSource:0}: Error finding container 4221aaf723213d7e7fdd5081574ae1e4de2356c46fb5c4a067cd8823b93fc660: Status 404 returned error can't find the container with id 4221aaf723213d7e7fdd5081574ae1e4de2356c46fb5c4a067cd8823b93fc660 Jul 09 14:09:29.605296 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.605257 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cndlx"] Jul 09 14:09:29.610458 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:29.610416 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfae5e3_218f_41e7_9c62_258cabbb800e.slice/crio-e68b3d14ee390519e46f06683b3a43342c017ebdcf1bace479603eecc396a191 WatchSource:0}: Error finding container e68b3d14ee390519e46f06683b3a43342c017ebdcf1bace479603eecc396a191: Status 404 returned error can't find the container with id e68b3d14ee390519e46f06683b3a43342c017ebdcf1bace479603eecc396a191 Jul 09 14:09:29.954768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.951808 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" event={"ID":"b8fa2904-5d6d-4615-b349-46793ed579eb","Type":"ContainerStarted","Data":"4221aaf723213d7e7fdd5081574ae1e4de2356c46fb5c4a067cd8823b93fc660"} Jul 09 14:09:29.954768 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:29.954385 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" event={"ID":"4cfae5e3-218f-41e7-9c62-258cabbb800e","Type":"ContainerStarted","Data":"e68b3d14ee390519e46f06683b3a43342c017ebdcf1bace479603eecc396a191"} Jul 09 14:09:33.881616 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:33.881582 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:09:33.882000 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:33.881645 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:09:34.995254 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.994862 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" event={"ID":"b8fa2904-5d6d-4615-b349-46793ed579eb","Type":"ContainerStarted","Data":"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033"} Jul 09 14:09:34.995254 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.995180 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:34.996967 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.996810 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-bhpkv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:09:34.996967 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.996882 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:09:34.997167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.997134 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" event={"ID":"3c70fa9f-1b94-4e60-aec5-3045950aa46d","Type":"ContainerStarted","Data":"36272dacb5dd2d843f52647bc649ad90510724dca58295bcb65e20fad116fe8a"} Jul 09 14:09:34.997319 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.997295 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:34.999651 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:34.999288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" event={"ID":"7eb16642-3fdf-4ffe-bdab-76d0510a93ef","Type":"ContainerStarted","Data":"3b44d106dd5c142cd65825ed3ebf5672c0ab6f74dea30b305e93351304343da3"} Jul 09 14:09:35.019611 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:35.019544 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" podStartSLOduration=2.579705215 podStartE2EDuration="7.019524423s" totalImagesPullingTime="4.439819208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:35.017902768 +0000 UTC m=+436.656577566" watchObservedRunningTime="2026-07-09 14:09:35.019524423 +0000 UTC m=+436.658199196" Jul 09 14:09:35.037301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:35.037250 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79b6dccbd4-wkmbf" podStartSLOduration=2.759084646 podStartE2EDuration="8.037216008s" totalImagesPullingTime="5.278131362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:35.035357435 +0000 UTC m=+436.674032230" watchObservedRunningTime="2026-07-09 14:09:35.037216008 +0000 UTC m=+436.675890783" Jul 09 14:09:35.057975 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:35.057910 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" podStartSLOduration=2.380958391 podStartE2EDuration="8.0578892s" totalImagesPullingTime="5.676930809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:35.057457311 +0000 UTC m=+436.696132128" watchObservedRunningTime="2026-07-09 14:09:35.0578892 +0000 UTC m=+436.696563973" Jul 09 14:09:36.007412 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:36.007366 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" event={"ID":"4cfae5e3-218f-41e7-9c62-258cabbb800e","Type":"ContainerStarted","Data":"57b4383b8ad8484c48daa096f7b1300d3a544c8aa3ea573d3b99218bc39647b1"} Jul 09 14:09:36.007863 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:36.007706 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:09:36.008731 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:36.008711 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:36.029751 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:36.029699 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" podStartSLOduration=2.380468766 podStartE2EDuration="8.029682495s" totalImagesPullingTime="5.649213729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:36.027719797 +0000 UTC m=+437.666394603" watchObservedRunningTime="2026-07-09 14:09:36.029682495 +0000 UTC m=+437.668357267" Jul 09 14:09:39.147022 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.146989 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr"] Jul 09 14:09:39.151080 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.151055 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42"] Jul 09 14:09:39.151209 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.151186 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.154486 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.154462 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-h6s8s\"" Jul 09 14:09:39.154620 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.154462 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 14:09:39.154719 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.154699 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.162403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.162380 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-47tll\"" Jul 09 14:09:39.162767 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.162748 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 14:09:39.165609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.165589 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr"] Jul 09 14:09:39.168621 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.168597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42"] Jul 09 14:09:39.220727 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.220679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbwcz\" (UniqueName: \"kubernetes.io/projected/8dc5e1be-05a1-4a77-9a72-737df6ac5475-kube-api-access-gbwcz\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.220912 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.220786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f428dd9b-c596-4ff0-8061-721507aca043-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.220912 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.220818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbct\" (UniqueName: \"kubernetes.io/projected/f428dd9b-c596-4ff0-8061-721507aca043-kube-api-access-6lbct\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.220912 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.220841 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc5e1be-05a1-4a77-9a72-737df6ac5475-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.322128 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.322084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwcz\" (UniqueName: \"kubernetes.io/projected/8dc5e1be-05a1-4a77-9a72-737df6ac5475-kube-api-access-gbwcz\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.322337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.322174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f428dd9b-c596-4ff0-8061-721507aca043-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.322337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.322204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbct\" (UniqueName: \"kubernetes.io/projected/f428dd9b-c596-4ff0-8061-721507aca043-kube-api-access-6lbct\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.322337 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.322248 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc5e1be-05a1-4a77-9a72-737df6ac5475-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.325467 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.325439 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc5e1be-05a1-4a77-9a72-737df6ac5475-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.325467 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.325465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f428dd9b-c596-4ff0-8061-721507aca043-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.333198 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.333163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbwcz\" (UniqueName: \"kubernetes.io/projected/8dc5e1be-05a1-4a77-9a72-737df6ac5475-kube-api-access-gbwcz\") pod \"llmisvc-controller-manager-777c6bd7d5-q8q42\" (UID: \"8dc5e1be-05a1-4a77-9a72-737df6ac5475\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.333625 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.333596 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbct\" (UniqueName: \"kubernetes.io/projected/f428dd9b-c596-4ff0-8061-721507aca043-kube-api-access-6lbct\") pod \"kserve-controller-manager-d6f4b65b8-4ckzr\" (UID: \"f428dd9b-c596-4ff0-8061-721507aca043\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.465664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.465576 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:39.471845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.471820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:39.636943 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.636864 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr"] Jul 09 14:09:39.642066 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:39.641878 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf428dd9b_c596_4ff0_8061_721507aca043.slice/crio-342e54a4c5b35043bfe0de2b93e8ead8aabd749db57feac9ead763b7ca3204e1 WatchSource:0}: Error finding container 342e54a4c5b35043bfe0de2b93e8ead8aabd749db57feac9ead763b7ca3204e1: Status 404 returned error can't find the container with id 342e54a4c5b35043bfe0de2b93e8ead8aabd749db57feac9ead763b7ca3204e1 Jul 09 14:09:39.661026 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:39.660100 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42"] Jul 09 14:09:39.665464 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:09:39.665409 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8dc5e1be_05a1_4a77_9a72_737df6ac5475.slice/crio-49dd7c87dc0230e1fc74662cc2a4acc9bebdccaa64c84e1ce1a2c2b4de2f2405 WatchSource:0}: Error finding container 49dd7c87dc0230e1fc74662cc2a4acc9bebdccaa64c84e1ce1a2c2b4de2f2405: Status 404 returned error can't find the container with id 49dd7c87dc0230e1fc74662cc2a4acc9bebdccaa64c84e1ce1a2c2b4de2f2405 Jul 09 14:09:40.032277 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:40.032240 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" event={"ID":"8dc5e1be-05a1-4a77-9a72-737df6ac5475","Type":"ContainerStarted","Data":"49dd7c87dc0230e1fc74662cc2a4acc9bebdccaa64c84e1ce1a2c2b4de2f2405"} Jul 09 14:09:40.033313 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:40.033287 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" event={"ID":"f428dd9b-c596-4ff0-8061-721507aca043","Type":"ContainerStarted","Data":"342e54a4c5b35043bfe0de2b93e8ead8aabd749db57feac9ead763b7ca3204e1"} Jul 09 14:09:42.152399 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.152292 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:42.152789 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.152527 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerName="discovery" containerID="cri-o://78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033" gracePeriod=30 Jul 09 14:09:42.470294 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.470261 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:42.554419 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554381 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554438 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554473 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d89bv\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554496 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554547 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554573 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554603 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig\") pod \"b8fa2904-5d6d-4615-b349-46793ed579eb\" (UID: \"b8fa2904-5d6d-4615-b349-46793ed579eb\") " Jul 09 14:09:42.554929 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554823 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:09:42.554981 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.554958 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-ca-configmap\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.556995 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.556936 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs" (OuterVolumeSpecName: "local-certs") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:09:42.557315 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.557249 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:09:42.557315 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.557270 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts" (OuterVolumeSpecName: "cacerts") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:09:42.557487 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.557351 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv" (OuterVolumeSpecName: "kube-api-access-d89bv") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "kube-api-access-d89bv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:09:42.557487 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.557354 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:09:42.557911 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.557883 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b8fa2904-5d6d-4615-b349-46793ed579eb" (UID: "b8fa2904-5d6d-4615-b349-46793ed579eb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656518 2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-kubeconfig\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656554 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-csr-dns-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656571 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d89bv\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-kube-api-access-d89bv\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656586 2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8fa2904-5d6d-4615-b349-46793ed579eb-local-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656600 2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8fa2904-5d6d-4615-b349-46793ed579eb-cacerts\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:42.657081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:42.656613 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8fa2904-5d6d-4615-b349-46793ed579eb-istio-token\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:09:43.054508 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.054471 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" event={"ID":"f428dd9b-c596-4ff0-8061-721507aca043","Type":"ContainerStarted","Data":"4bd26e50118f9479bb8db0d37fdea1ce34bc0bccd88b4c643df264a075dd64b2"} Jul 09 14:09:43.054698 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.054563 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:09:43.056127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.056095 2576 generic.go:356] "Generic (PLEG): container finished" podID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerID="78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033" exitCode=0 Jul 09 14:09:43.056290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.056146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" event={"ID":"b8fa2904-5d6d-4615-b349-46793ed579eb","Type":"ContainerDied","Data":"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033"} Jul 09 14:09:43.056290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.056159 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" Jul 09 14:09:43.056290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.056185 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv" event={"ID":"b8fa2904-5d6d-4615-b349-46793ed579eb","Type":"ContainerDied","Data":"4221aaf723213d7e7fdd5081574ae1e4de2356c46fb5c4a067cd8823b93fc660"} Jul 09 14:09:43.056290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.056212 2576 scope.go:117] "RemoveContainer" containerID="78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033" Jul 09 14:09:43.065454 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.065436 2576 scope.go:117] "RemoveContainer" containerID="78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033" Jul 09 14:09:43.065730 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:09:43.065710 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033\": container with ID starting with 78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033 not found: ID does not exist" containerID="78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033" Jul 09 14:09:43.065775 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.065737 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033"} err="failed to get container status \"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033\": rpc error: code = NotFound desc = could not find container \"78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033\": container with ID starting with 78274cffb93287ed7790f0e5e629cb08ad9a0f63e206dfecb6c706f20750a033 not found: ID does not exist" Jul 09 14:09:43.076144 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.076093 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" podStartSLOduration=1.426000717 podStartE2EDuration="4.076078614s" totalImagesPullingTime="2.650077897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:43.073175945 +0000 UTC m=+444.711850738" watchObservedRunningTime="2026-07-09 14:09:43.076078614 +0000 UTC m=+444.714753386" Jul 09 14:09:43.090503 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.090470 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:43.094907 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:43.094879 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhpkv"] Jul 09 14:09:44.895839 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:44.895807 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" path="/var/lib/kubelet/pods/b8fa2904-5d6d-4615-b349-46793ed579eb/volumes" Jul 09 14:09:46.009624 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:46.009590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tfxx6" Jul 09 14:09:46.076432 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:46.076382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" event={"ID":"8dc5e1be-05a1-4a77-9a72-737df6ac5475","Type":"ContainerStarted","Data":"c83352af3e84385fef62a778d05de682f3a8d98f0924cc34841b81b4fdec6777"} Jul 09 14:09:46.076612 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:46.076452 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:09:46.103887 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:46.103834 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" podStartSLOduration=1.8199666429999999 podStartE2EDuration="7.103817106s" totalImagesPullingTime="5.283850463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:46.102032733 +0000 UTC m=+447.740707535" watchObservedRunningTime="2026-07-09 14:09:46.103817106 +0000 UTC m=+447.742491879" Jul 09 14:09:47.017879 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:09:47.017844 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cndlx" Jul 09 14:10:14.067537 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:14.067492 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ckzr" Jul 09 14:10:17.084750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:17.084718 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q8q42" Jul 09 14:10:28.750169 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.750128 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:28.750595 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.750541 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerName="discovery" Jul 09 14:10:28.750595 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.750554 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerName="discovery" Jul 09 14:10:28.750668 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.750649 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8fa2904-5d6d-4615-b349-46793ed579eb" containerName="discovery" Jul 09 14:10:28.759530 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.759500 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:28.763075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.762531 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:10:28.763075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.762657 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:10:28.763075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.762784 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bfnq9\"" Jul 09 14:10:28.764399 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.764370 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:28.898941 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:28.898900 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txkgl\" (UniqueName: \"kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl\") pod \"kuadrant-operator-catalog-m28vx\" (UID: \"db87770a-24bc-48a1-9d42-3b9f75041404\") " pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:29.000650 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.000552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txkgl\" (UniqueName: \"kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl\") pod \"kuadrant-operator-catalog-m28vx\" (UID: \"db87770a-24bc-48a1-9d42-3b9f75041404\") " pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:29.008881 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.008852 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txkgl\" (UniqueName: \"kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl\") pod \"kuadrant-operator-catalog-m28vx\" (UID: \"db87770a-24bc-48a1-9d42-3b9f75041404\") " pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:29.071505 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.071471 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:29.112864 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.112813 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:29.209004 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.208972 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:29.210324 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:29.210294 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb87770a_24bc_48a1_9d42_3b9f75041404.slice/crio-1e96ae402539acd1d972eafdfcf5669dc67f0d396955a539e37155b5e962b4e2 WatchSource:0}: Error finding container 1e96ae402539acd1d972eafdfcf5669dc67f0d396955a539e37155b5e962b4e2: Status 404 returned error can't find the container with id 1e96ae402539acd1d972eafdfcf5669dc67f0d396955a539e37155b5e962b4e2 Jul 09 14:10:29.313904 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.313864 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" event={"ID":"db87770a-24bc-48a1-9d42-3b9f75041404","Type":"ContainerStarted","Data":"1e96ae402539acd1d972eafdfcf5669dc67f0d396955a539e37155b5e962b4e2"} Jul 09 14:10:29.325025 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.324992 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgv9q"] Jul 09 14:10:29.330424 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.330404 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:29.337104 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.337071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgv9q"] Jul 09 14:10:29.506485 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.506440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82w6c\" (UniqueName: \"kubernetes.io/projected/fbe80fa5-49b9-4941-844f-44293610551c-kube-api-access-82w6c\") pod \"kuadrant-operator-catalog-jgv9q\" (UID: \"fbe80fa5-49b9-4941-844f-44293610551c\") " pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:29.607919 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.607818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82w6c\" (UniqueName: \"kubernetes.io/projected/fbe80fa5-49b9-4941-844f-44293610551c-kube-api-access-82w6c\") pod \"kuadrant-operator-catalog-jgv9q\" (UID: \"fbe80fa5-49b9-4941-844f-44293610551c\") " pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:29.616122 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.616085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82w6c\" (UniqueName: \"kubernetes.io/projected/fbe80fa5-49b9-4941-844f-44293610551c-kube-api-access-82w6c\") pod \"kuadrant-operator-catalog-jgv9q\" (UID: \"fbe80fa5-49b9-4941-844f-44293610551c\") " pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:29.641765 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.641729 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:29.777456 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:29.777427 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgv9q"] Jul 09 14:10:29.779278 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:29.779247 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe80fa5_49b9_4941_844f_44293610551c.slice/crio-d38b98bc88222fca54a0bce117d549986c45535ecdce1d7e20eb012b62ae09c5 WatchSource:0}: Error finding container d38b98bc88222fca54a0bce117d549986c45535ecdce1d7e20eb012b62ae09c5: Status 404 returned error can't find the container with id d38b98bc88222fca54a0bce117d549986c45535ecdce1d7e20eb012b62ae09c5 Jul 09 14:10:30.320221 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:30.320174 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" event={"ID":"fbe80fa5-49b9-4941-844f-44293610551c","Type":"ContainerStarted","Data":"d38b98bc88222fca54a0bce117d549986c45535ecdce1d7e20eb012b62ae09c5"} Jul 09 14:10:32.334613 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.334574 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" event={"ID":"fbe80fa5-49b9-4941-844f-44293610551c","Type":"ContainerStarted","Data":"57c6d88dd6f67c02759c02b9127b5435412f636c983787fccb820c11075d7f7c"} Jul 09 14:10:32.336220 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.336192 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" event={"ID":"db87770a-24bc-48a1-9d42-3b9f75041404","Type":"ContainerStarted","Data":"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a"} Jul 09 14:10:32.336379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.336266 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" podUID="db87770a-24bc-48a1-9d42-3b9f75041404" containerName="registry-server" containerID="cri-o://4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a" gracePeriod=2 Jul 09 14:10:32.364212 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.364161 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" podStartSLOduration=1.177664905 podStartE2EDuration="3.364144854s" totalImagesPullingTime="2.186479949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:10:32.363504434 +0000 UTC m=+494.002179207" watchObservedRunningTime="2026-07-09 14:10:32.364144854 +0000 UTC m=+494.002819605" Jul 09 14:10:32.611564 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.611531 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:32.736636 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.736604 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txkgl\" (UniqueName: \"kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl\") pod \"db87770a-24bc-48a1-9d42-3b9f75041404\" (UID: \"db87770a-24bc-48a1-9d42-3b9f75041404\") " Jul 09 14:10:32.738850 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.738821 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl" (OuterVolumeSpecName: "kube-api-access-txkgl") pod "db87770a-24bc-48a1-9d42-3b9f75041404" (UID: "db87770a-24bc-48a1-9d42-3b9f75041404"). InnerVolumeSpecName "kube-api-access-txkgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:10:32.838079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:32.838035 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txkgl\" (UniqueName: \"kubernetes.io/projected/db87770a-24bc-48a1-9d42-3b9f75041404-kube-api-access-txkgl\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:33.346137 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.346099 2576 generic.go:356] "Generic (PLEG): container finished" podID="db87770a-24bc-48a1-9d42-3b9f75041404" containerID="4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a" exitCode=0 Jul 09 14:10:33.346544 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.346145 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" Jul 09 14:10:33.346544 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.346180 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" event={"ID":"db87770a-24bc-48a1-9d42-3b9f75041404","Type":"ContainerDied","Data":"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a"} Jul 09 14:10:33.346544 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.346242 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m28vx" event={"ID":"db87770a-24bc-48a1-9d42-3b9f75041404","Type":"ContainerDied","Data":"1e96ae402539acd1d972eafdfcf5669dc67f0d396955a539e37155b5e962b4e2"} Jul 09 14:10:33.346544 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.346264 2576 scope.go:117] "RemoveContainer" containerID="4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a" Jul 09 14:10:33.355738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.355716 2576 scope.go:117] "RemoveContainer" containerID="4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a" Jul 09 14:10:33.356022 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:10:33.356000 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a\": container with ID starting with 4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a not found: ID does not exist" containerID="4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a" Jul 09 14:10:33.356095 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.356034 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a"} err="failed to get container status \"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a\": rpc error: code = NotFound desc = could not find container \"4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a\": container with ID starting with 4f885a836f7240a2b58ef72cbd433493ef62840cd1f5a3f512656f6e585e2c0a not found: ID does not exist" Jul 09 14:10:33.364019 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.363990 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:33.368098 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:33.368071 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m28vx"] Jul 09 14:10:34.894611 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:34.894575 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db87770a-24bc-48a1-9d42-3b9f75041404" path="/var/lib/kubelet/pods/db87770a-24bc-48a1-9d42-3b9f75041404/volumes" Jul 09 14:10:39.642590 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:39.642543 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:39.643087 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:39.642617 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:39.665886 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:39.665857 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:40.407936 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:40.407900 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jgv9q" Jul 09 14:10:43.937563 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.937525 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc"] Jul 09 14:10:43.937943 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.937933 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db87770a-24bc-48a1-9d42-3b9f75041404" containerName="registry-server" Jul 09 14:10:43.937943 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.937943 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db87770a-24bc-48a1-9d42-3b9f75041404" containerName="registry-server" Jul 09 14:10:43.938031 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.938021 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db87770a-24bc-48a1-9d42-3b9f75041404" containerName="registry-server" Jul 09 14:10:43.941434 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.941417 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:43.943949 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.943924 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7nrjf\"" Jul 09 14:10:43.949952 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:43.949927 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc"] Jul 09 14:10:44.033901 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.033863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.034083 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.033917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.034083 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.034002 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh2qf\" (UniqueName: \"kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.135519 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.135477 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.135672 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.135542 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.135672 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.135596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh2qf\" (UniqueName: \"kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.135915 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.135893 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.135968 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.135945 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.144853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.144812 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh2qf\" (UniqueName: \"kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.252074 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.251978 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:44.396475 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.396444 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc"] Jul 09 14:10:44.398698 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:44.398666 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2b1bc9_58fc_48ab_9e90_342afbb87cfb.slice/crio-cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87 WatchSource:0}: Error finding container cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87: Status 404 returned error can't find the container with id cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87 Jul 09 14:10:44.408601 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.408574 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerStarted","Data":"cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87"} Jul 09 14:10:44.738527 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.738492 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr"] Jul 09 14:10:44.742438 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.742413 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.752215 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.752185 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr"] Jul 09 14:10:44.842139 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.842102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.842358 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.842225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.842358 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.842302 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtdj\" (UniqueName: \"kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.943671 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.943635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.944155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.943739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.944155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.943771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtdj\" (UniqueName: \"kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.944155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.944107 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.944155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.944109 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:44.953130 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:44.953109 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtdj\" (UniqueName: \"kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:45.052452 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.052420 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:45.138301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.138265 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp"] Jul 09 14:10:45.156905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.155616 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.159948 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.159919 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp"] Jul 09 14:10:45.203531 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.203501 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr"] Jul 09 14:10:45.205788 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:45.205749 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f99786_912b_4767_9be1_c79fa08db0a7.slice/crio-b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51 WatchSource:0}: Error finding container b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51: Status 404 returned error can't find the container with id b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51 Jul 09 14:10:45.250720 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.250693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.251156 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.250746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.251282 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.251259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgxk7\" (UniqueName: \"kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.352286 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.352216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.352469 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.352323 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.352469 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.352399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgxk7\" (UniqueName: \"kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.352661 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.352643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.352718 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.352700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.362521 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.362455 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgxk7\" (UniqueName: \"kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.415937 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.415897 2576 generic.go:356] "Generic (PLEG): container finished" podID="64f99786-912b-4767-9be1-c79fa08db0a7" containerID="2d752b330e0918d3c1ccdc3762128747116aa5bfa899f9ecaf56a6d3d4bcf2be" exitCode=0 Jul 09 14:10:45.416127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.415981 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerDied","Data":"2d752b330e0918d3c1ccdc3762128747116aa5bfa899f9ecaf56a6d3d4bcf2be"} Jul 09 14:10:45.416127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.416040 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerStarted","Data":"b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51"} Jul 09 14:10:45.417783 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.417758 2576 generic.go:356] "Generic (PLEG): container finished" podID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerID="527dad33980b822d7fc834071d28e241f273ef47e128d2f75a0abfabe0aa673e" exitCode=0 Jul 09 14:10:45.417903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.417789 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerDied","Data":"527dad33980b822d7fc834071d28e241f273ef47e128d2f75a0abfabe0aa673e"} Jul 09 14:10:45.472559 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.472518 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:45.538279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.538209 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg"] Jul 09 14:10:45.543290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.543262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.553521 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.553492 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg"] Jul 09 14:10:45.619160 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.619135 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp"] Jul 09 14:10:45.620816 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:45.620785 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1494dff_e2cc_40cd_a938_2ba2d420f41d.slice/crio-99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35 WatchSource:0}: Error finding container 99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35: Status 404 returned error can't find the container with id 99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35 Jul 09 14:10:45.655330 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.655300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvc9w\" (UniqueName: \"kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.655477 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.655392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.655672 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.655506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.757004 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.756961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.757206 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.757025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvc9w\" (UniqueName: \"kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.757206 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.757075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.757428 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.757403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.757490 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.757436 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.766052 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.766020 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvc9w\" (UniqueName: \"kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.854721 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.854682 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:45.996386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:45.996356 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg"] Jul 09 14:10:45.998153 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:45.998127 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cf314c_e445_4804_983f_3b1f74649e13.slice/crio-4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7 WatchSource:0}: Error finding container 4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7: Status 404 returned error can't find the container with id 4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7 Jul 09 14:10:46.426218 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.426176 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerStarted","Data":"f30141348b3766a6cdd375ae9cecd2ceb4f971d0c75baf7015f2fc2787606782"} Jul 09 14:10:46.427965 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.427924 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerID="396d1c4b18f21be4154d120d0e99c9420069a86f9f09a6101d78675957f9241d" exitCode=0 Jul 09 14:10:46.428131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.428030 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" event={"ID":"f1494dff-e2cc-40cd-a938-2ba2d420f41d","Type":"ContainerDied","Data":"396d1c4b18f21be4154d120d0e99c9420069a86f9f09a6101d78675957f9241d"} Jul 09 14:10:46.428131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.428063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" event={"ID":"f1494dff-e2cc-40cd-a938-2ba2d420f41d","Type":"ContainerStarted","Data":"99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35"} Jul 09 14:10:46.430403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.430372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerStarted","Data":"76cedaa32c4baa993230365845d60813523bb71ab8d51f2b0dc46f1dd1c87799"} Jul 09 14:10:46.432270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.432242 2576 generic.go:356] "Generic (PLEG): container finished" podID="41cf314c-e445-4804-983f-3b1f74649e13" containerID="6ce8dd399962e0a5b3c5c4500570b800d82de050ac3c583dc5dd3cf6059ed260" exitCode=0 Jul 09 14:10:46.432446 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.432317 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerDied","Data":"6ce8dd399962e0a5b3c5c4500570b800d82de050ac3c583dc5dd3cf6059ed260"} Jul 09 14:10:46.432566 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:46.432551 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerStarted","Data":"4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7"} Jul 09 14:10:47.439783 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.439752 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerID="f34e895986af836cb64e3a11846134b8268c07ab205b5b94d8434771843bea35" exitCode=0 Jul 09 14:10:47.440200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.439831 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" event={"ID":"f1494dff-e2cc-40cd-a938-2ba2d420f41d","Type":"ContainerDied","Data":"f34e895986af836cb64e3a11846134b8268c07ab205b5b94d8434771843bea35"} Jul 09 14:10:47.442082 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.442057 2576 generic.go:356] "Generic (PLEG): container finished" podID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerID="76cedaa32c4baa993230365845d60813523bb71ab8d51f2b0dc46f1dd1c87799" exitCode=0 Jul 09 14:10:47.442186 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.442144 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerDied","Data":"76cedaa32c4baa993230365845d60813523bb71ab8d51f2b0dc46f1dd1c87799"} Jul 09 14:10:47.444404 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.444382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerStarted","Data":"aae068afc570a52da79b9513b75a233153e4fae483347311c2705bc737607639"} Jul 09 14:10:47.446476 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.446451 2576 generic.go:356] "Generic (PLEG): container finished" podID="64f99786-912b-4767-9be1-c79fa08db0a7" containerID="f30141348b3766a6cdd375ae9cecd2ceb4f971d0c75baf7015f2fc2787606782" exitCode=0 Jul 09 14:10:47.446565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:47.446489 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerDied","Data":"f30141348b3766a6cdd375ae9cecd2ceb4f971d0c75baf7015f2fc2787606782"} Jul 09 14:10:48.454918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.454829 2576 generic.go:356] "Generic (PLEG): container finished" podID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerID="79cdf276f50d98724addb0a701b27c30b129dd8d7fc275b200e127fb0deaef71" exitCode=0 Jul 09 14:10:48.455392 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.454914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerDied","Data":"79cdf276f50d98724addb0a701b27c30b129dd8d7fc275b200e127fb0deaef71"} Jul 09 14:10:48.456648 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.456620 2576 generic.go:356] "Generic (PLEG): container finished" podID="41cf314c-e445-4804-983f-3b1f74649e13" containerID="aae068afc570a52da79b9513b75a233153e4fae483347311c2705bc737607639" exitCode=0 Jul 09 14:10:48.456753 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.456702 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerDied","Data":"aae068afc570a52da79b9513b75a233153e4fae483347311c2705bc737607639"} Jul 09 14:10:48.459020 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.458995 2576 generic.go:356] "Generic (PLEG): container finished" podID="64f99786-912b-4767-9be1-c79fa08db0a7" containerID="af5907e8b808df4ac83988529080a907a66e8d33c195435a65219c03ba3b0286" exitCode=0 Jul 09 14:10:48.459127 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.459069 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerDied","Data":"af5907e8b808df4ac83988529080a907a66e8d33c195435a65219c03ba3b0286"} Jul 09 14:10:48.461025 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.461005 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerID="8f4a4ef1ecbee935feb4f97da2c3490cc0ff58d7611d5b037af7dff8eacc26ee" exitCode=0 Jul 09 14:10:48.461096 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:48.461033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" event={"ID":"f1494dff-e2cc-40cd-a938-2ba2d420f41d","Type":"ContainerDied","Data":"8f4a4ef1ecbee935feb4f97da2c3490cc0ff58d7611d5b037af7dff8eacc26ee"} Jul 09 14:10:49.469169 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.469136 2576 generic.go:356] "Generic (PLEG): container finished" podID="41cf314c-e445-4804-983f-3b1f74649e13" containerID="f261840919effd6ee953b56f2a0502248f448a6225878c6b2263159087f4aa23" exitCode=0 Jul 09 14:10:49.469629 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.469240 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerDied","Data":"f261840919effd6ee953b56f2a0502248f448a6225878c6b2263159087f4aa23"} Jul 09 14:10:49.658354 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.658323 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:49.662474 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.662437 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:49.667038 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.667012 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:49.694046 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.693988 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util\") pod \"64f99786-912b-4767-9be1-c79fa08db0a7\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " Jul 09 14:10:49.694863 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.694831 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util\") pod \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " Jul 09 14:10:49.695014 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.694880 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle\") pod \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " Jul 09 14:10:49.695014 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.694948 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh2qf\" (UniqueName: \"kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf\") pod \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " Jul 09 14:10:49.695014 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.694995 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle\") pod \"64f99786-912b-4767-9be1-c79fa08db0a7\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " Jul 09 14:10:49.695177 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.695032 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util\") pod \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\" (UID: \"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb\") " Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.695651 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle\") pod \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.696005 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle" (OuterVolumeSpecName: "bundle") pod "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" (UID: "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.696368 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle" (OuterVolumeSpecName: "bundle") pod "64f99786-912b-4767-9be1-c79fa08db0a7" (UID: "64f99786-912b-4767-9be1-c79fa08db0a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.696587 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle" (OuterVolumeSpecName: "bundle") pod "f1494dff-e2cc-40cd-a938-2ba2d420f41d" (UID: "f1494dff-e2cc-40cd-a938-2ba2d420f41d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.696919 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mtdj\" (UniqueName: \"kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj\") pod \"64f99786-912b-4767-9be1-c79fa08db0a7\" (UID: \"64f99786-912b-4767-9be1-c79fa08db0a7\") " Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.696995 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgxk7\" (UniqueName: \"kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7\") pod \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\" (UID: \"f1494dff-e2cc-40cd-a938-2ba2d420f41d\") " Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.699043 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj" (OuterVolumeSpecName: "kube-api-access-9mtdj") pod "64f99786-912b-4767-9be1-c79fa08db0a7" (UID: "64f99786-912b-4767-9be1-c79fa08db0a7"). InnerVolumeSpecName "kube-api-access-9mtdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.699334 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.699356 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.699388 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.699368 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.701209 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.701180 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util" (OuterVolumeSpecName: "util") pod "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" (UID: "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.702120 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.702093 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7" (OuterVolumeSpecName: "kube-api-access-zgxk7") pod "f1494dff-e2cc-40cd-a938-2ba2d420f41d" (UID: "f1494dff-e2cc-40cd-a938-2ba2d420f41d"). InnerVolumeSpecName "kube-api-access-zgxk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:10:49.702582 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.702552 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util" (OuterVolumeSpecName: "util") pod "64f99786-912b-4767-9be1-c79fa08db0a7" (UID: "64f99786-912b-4767-9be1-c79fa08db0a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.703270 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.703217 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf" (OuterVolumeSpecName: "kube-api-access-lh2qf") pod "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" (UID: "8a2b1bc9-58fc-48ab-9e90-342afbb87cfb"). InnerVolumeSpecName "kube-api-access-lh2qf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:10:49.704438 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.704414 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util" (OuterVolumeSpecName: "util") pod "f1494dff-e2cc-40cd-a938-2ba2d420f41d" (UID: "f1494dff-e2cc-40cd-a938-2ba2d420f41d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800114 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800144 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mtdj\" (UniqueName: \"kubernetes.io/projected/64f99786-912b-4767-9be1-c79fa08db0a7-kube-api-access-9mtdj\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800154 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgxk7\" (UniqueName: \"kubernetes.io/projected/f1494dff-e2cc-40cd-a938-2ba2d420f41d-kube-api-access-zgxk7\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800163 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64f99786-912b-4767-9be1-c79fa08db0a7-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800171 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1494dff-e2cc-40cd-a938-2ba2d420f41d-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:49.800200 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:49.800179 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh2qf\" (UniqueName: \"kubernetes.io/projected/8a2b1bc9-58fc-48ab-9e90-342afbb87cfb-kube-api-access-lh2qf\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:50.476185 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.476150 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" Jul 09 14:10:50.476596 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.476155 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr" event={"ID":"64f99786-912b-4767-9be1-c79fa08db0a7","Type":"ContainerDied","Data":"b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51"} Jul 09 14:10:50.476596 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.476316 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b855b5b03ad07092d704b377c0fafa3ab6847aae1340781abc1a03e4f6336c51" Jul 09 14:10:50.478030 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.478005 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" Jul 09 14:10:50.478167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.478034 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp" event={"ID":"f1494dff-e2cc-40cd-a938-2ba2d420f41d","Type":"ContainerDied","Data":"99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35"} Jul 09 14:10:50.478167 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.478081 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99a0d86ad6b64b2e136b5b5954fcc1bf942e50e0a606c3c0ae77b82919e1bc35" Jul 09 14:10:50.479986 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.479963 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" event={"ID":"8a2b1bc9-58fc-48ab-9e90-342afbb87cfb","Type":"ContainerDied","Data":"cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87"} Jul 09 14:10:50.480096 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.480005 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd93c69a293734ac2e26d6b50250f8d6e8c005058e53dc5d5ba2d0b6d1158e87" Jul 09 14:10:50.480096 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.479972 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc" Jul 09 14:10:50.621357 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.621326 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:50.708693 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.708659 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util\") pod \"41cf314c-e445-4804-983f-3b1f74649e13\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " Jul 09 14:10:50.708855 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.708733 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle\") pod \"41cf314c-e445-4804-983f-3b1f74649e13\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " Jul 09 14:10:50.708855 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.708784 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvc9w\" (UniqueName: \"kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w\") pod \"41cf314c-e445-4804-983f-3b1f74649e13\" (UID: \"41cf314c-e445-4804-983f-3b1f74649e13\") " Jul 09 14:10:50.709290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.709226 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle" (OuterVolumeSpecName: "bundle") pod "41cf314c-e445-4804-983f-3b1f74649e13" (UID: "41cf314c-e445-4804-983f-3b1f74649e13"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:50.710833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.710810 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w" (OuterVolumeSpecName: "kube-api-access-tvc9w") pod "41cf314c-e445-4804-983f-3b1f74649e13" (UID: "41cf314c-e445-4804-983f-3b1f74649e13"). InnerVolumeSpecName "kube-api-access-tvc9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:10:50.715458 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.715430 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util" (OuterVolumeSpecName: "util") pod "41cf314c-e445-4804-983f-3b1f74649e13" (UID: "41cf314c-e445-4804-983f-3b1f74649e13"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:10:50.809723 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.809685 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvc9w\" (UniqueName: \"kubernetes.io/projected/41cf314c-e445-4804-983f-3b1f74649e13-kube-api-access-tvc9w\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:50.809872 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.809724 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-util\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:50.809872 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:50.809736 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cf314c-e445-4804-983f-3b1f74649e13-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:10:51.011776 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.011686 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b9db479fb-wc9jr"] Jul 09 14:10:51.012313 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012278 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="util" Jul 09 14:10:51.012313 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012315 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012332 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="extract" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012340 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="extract" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012356 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012364 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012379 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="pull" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012387 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="pull" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012395 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="pull" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012403 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="pull" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012415 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012475 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012483 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012491 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="util" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012504 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="extract" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012513 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="extract" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012529 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="extract" Jul 09 14:10:51.012529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012537 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012548 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012556 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012567 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="pull" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012577 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="pull" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012603 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="pull" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012611 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="pull" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012745 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41cf314c-e445-4804-983f-3b1f74649e13" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012763 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1494dff-e2cc-40cd-a938-2ba2d420f41d" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012776 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f99786-912b-4767-9be1-c79fa08db0a7" containerName="extract" Jul 09 14:10:51.013279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.012785 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2b1bc9-58fc-48ab-9e90-342afbb87cfb" containerName="extract" Jul 09 14:10:51.017323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.017300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.024863 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.024832 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b9db479fb-wc9jr"] Jul 09 14:10:51.113928 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.113823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-service-ca\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.113928 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.113876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.114148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.113971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-oauth-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.114148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.114056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndfkd\" (UniqueName: \"kubernetes.io/projected/79eee603-324d-49c0-8c47-ac1a2e94dc1b-kube-api-access-ndfkd\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.114148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.114099 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.114148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.114143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-oauth-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.114379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.114218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-trusted-ca-bundle\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215064 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-oauth-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215250 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-trusted-ca-bundle\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215250 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-service-ca\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215250 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215250 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215181 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-oauth-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215476 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndfkd\" (UniqueName: \"kubernetes.io/projected/79eee603-324d-49c0-8c47-ac1a2e94dc1b-kube-api-access-ndfkd\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215476 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.215951 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.215925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-service-ca\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.216066 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.216020 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-oauth-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.216139 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.216096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.216282 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.216129 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79eee603-324d-49c0-8c47-ac1a2e94dc1b-trusted-ca-bundle\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.217716 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.217690 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-serving-cert\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.217896 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.217880 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79eee603-324d-49c0-8c47-ac1a2e94dc1b-console-oauth-config\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.225378 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.225351 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndfkd\" (UniqueName: \"kubernetes.io/projected/79eee603-324d-49c0-8c47-ac1a2e94dc1b-kube-api-access-ndfkd\") pod \"console-b9db479fb-wc9jr\" (UID: \"79eee603-324d-49c0-8c47-ac1a2e94dc1b\") " pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.328298 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.328253 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:10:51.470159 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.470134 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b9db479fb-wc9jr"] Jul 09 14:10:51.472093 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:10:51.472071 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79eee603_324d_49c0_8c47_ac1a2e94dc1b.slice/crio-5ee3133a9bd3be30e74427d2b7db9c6c090bab96a3d0998c0ffe6e00f003abe7 WatchSource:0}: Error finding container 5ee3133a9bd3be30e74427d2b7db9c6c090bab96a3d0998c0ffe6e00f003abe7: Status 404 returned error can't find the container with id 5ee3133a9bd3be30e74427d2b7db9c6c090bab96a3d0998c0ffe6e00f003abe7 Jul 09 14:10:51.490204 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.490170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" event={"ID":"41cf314c-e445-4804-983f-3b1f74649e13","Type":"ContainerDied","Data":"4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7"} Jul 09 14:10:51.490592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.490187 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg" Jul 09 14:10:51.490592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.490255 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4975fcc01ccd9343e881ba5054e8793a79c6320e90dc33696881d148b90658a7" Jul 09 14:10:51.491539 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:51.491505 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b9db479fb-wc9jr" event={"ID":"79eee603-324d-49c0-8c47-ac1a2e94dc1b","Type":"ContainerStarted","Data":"5ee3133a9bd3be30e74427d2b7db9c6c090bab96a3d0998c0ffe6e00f003abe7"} Jul 09 14:10:52.498581 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:52.498543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b9db479fb-wc9jr" event={"ID":"79eee603-324d-49c0-8c47-ac1a2e94dc1b","Type":"ContainerStarted","Data":"3ced593f3d94ed344785e28766433cb68d300e29e87b63ef3fbbd213592ae788"} Jul 09 14:10:52.517285 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:10:52.517213 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b9db479fb-wc9jr" podStartSLOduration=2.517195296 podStartE2EDuration="2.517195296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:10:52.516561746 +0000 UTC m=+514.155236562" watchObservedRunningTime="2026-07-09 14:10:52.517195296 +0000 UTC m=+514.155870070" Jul 09 14:11:01.328442 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:01.328402 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:11:01.328859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:01.328469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:11:01.333552 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:01.333524 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:11:01.550281 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:01.550216 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b9db479fb-wc9jr" Jul 09 14:11:01.603883 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:01.603802 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:11:07.700140 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.700098 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw"] Jul 09 14:11:07.708474 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.708447 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:07.711464 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.711439 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dnc2d\"" Jul 09 14:11:07.717734 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.717704 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw"] Jul 09 14:11:07.883168 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.883128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9m8\" (UniqueName: \"kubernetes.io/projected/2d8ffc9c-25d5-419c-9433-39d3fa0eea27-kube-api-access-vb9m8\") pod \"authorino-operator-75cc8d5cb7-ss4nw\" (UID: \"2d8ffc9c-25d5-419c-9433-39d3fa0eea27\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:07.984423 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.984321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9m8\" (UniqueName: \"kubernetes.io/projected/2d8ffc9c-25d5-419c-9433-39d3fa0eea27-kube-api-access-vb9m8\") pod \"authorino-operator-75cc8d5cb7-ss4nw\" (UID: \"2d8ffc9c-25d5-419c-9433-39d3fa0eea27\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:07.998097 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:07.998066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9m8\" (UniqueName: \"kubernetes.io/projected/2d8ffc9c-25d5-419c-9433-39d3fa0eea27-kube-api-access-vb9m8\") pod \"authorino-operator-75cc8d5cb7-ss4nw\" (UID: \"2d8ffc9c-25d5-419c-9433-39d3fa0eea27\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:08.020212 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:08.020174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:08.166514 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:08.166488 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw"] Jul 09 14:11:08.168113 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:08.168081 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8ffc9c_25d5_419c_9433_39d3fa0eea27.slice/crio-9022f91d34c9b4b43ed46138538562371f2eb0c6feb2e5d04aa03e3477ac3324 WatchSource:0}: Error finding container 9022f91d34c9b4b43ed46138538562371f2eb0c6feb2e5d04aa03e3477ac3324: Status 404 returned error can't find the container with id 9022f91d34c9b4b43ed46138538562371f2eb0c6feb2e5d04aa03e3477ac3324 Jul 09 14:11:08.587008 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:08.586969 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" event={"ID":"2d8ffc9c-25d5-419c-9433-39d3fa0eea27","Type":"ContainerStarted","Data":"9022f91d34c9b4b43ed46138538562371f2eb0c6feb2e5d04aa03e3477ac3324"} Jul 09 14:11:10.600348 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:10.600307 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" event={"ID":"2d8ffc9c-25d5-419c-9433-39d3fa0eea27","Type":"ContainerStarted","Data":"dc4c46cee9c513448b2082639051d6f8c7bff645ac4c68f80de7dafc0528e978"} Jul 09 14:11:10.600817 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:10.600511 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:10.621340 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:10.621282 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" podStartSLOduration=1.512151644 podStartE2EDuration="3.621264912s" totalImagesPullingTime="2.109113268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:10.617715726 +0000 UTC m=+532.256390535" watchObservedRunningTime="2026-07-09 14:11:10.621264912 +0000 UTC m=+532.259939687" Jul 09 14:11:11.124641 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.124606 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj"] Jul 09 14:11:11.128522 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.128495 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:11.131561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.131534 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mvzsg\"" Jul 09 14:11:11.144330 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.144288 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj"] Jul 09 14:11:11.318126 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.318084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2bw5\" (UniqueName: \"kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5\") pod \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" (UID: \"b3e78290-588f-4795-b796-4c3443a1b817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:11.418772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.418654 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2bw5\" (UniqueName: \"kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5\") pod \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" (UID: \"b3e78290-588f-4795-b796-4c3443a1b817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:11.429468 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.429435 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2bw5\" (UniqueName: \"kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5\") pod \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" (UID: \"b3e78290-588f-4795-b796-4c3443a1b817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:11.438628 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.438596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:11.607301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:11.607273 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj"] Jul 09 14:11:11.609387 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:11.609360 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e78290_588f_4795_b796_4c3443a1b817.slice/crio-9c2b7415c1663bf6bbb3c63a450cd8e2596f8bb6458c4782c807055b985d44a7 WatchSource:0}: Error finding container 9c2b7415c1663bf6bbb3c63a450cd8e2596f8bb6458c4782c807055b985d44a7: Status 404 returned error can't find the container with id 9c2b7415c1663bf6bbb3c63a450cd8e2596f8bb6458c4782c807055b985d44a7 Jul 09 14:11:12.614971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:12.614932 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" event={"ID":"b3e78290-588f-4795-b796-4c3443a1b817","Type":"ContainerStarted","Data":"9c2b7415c1663bf6bbb3c63a450cd8e2596f8bb6458c4782c807055b985d44a7"} Jul 09 14:11:14.429383 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.429346 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx"] Jul 09 14:11:14.433219 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.433183 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:14.436051 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.436016 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 14:11:14.436162 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.436112 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fnvlv\"" Jul 09 14:11:14.444510 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.444484 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx"] Jul 09 14:11:14.450760 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.450721 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tfgt\" (UniqueName: \"kubernetes.io/projected/952b6856-5488-4439-8b8f-cd2d0cf63b4c-kube-api-access-9tfgt\") pod \"dns-operator-controller-manager-864dd78c4c-t9hzx\" (UID: \"952b6856-5488-4439-8b8f-cd2d0cf63b4c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:14.552360 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.552311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tfgt\" (UniqueName: \"kubernetes.io/projected/952b6856-5488-4439-8b8f-cd2d0cf63b4c-kube-api-access-9tfgt\") pod \"dns-operator-controller-manager-864dd78c4c-t9hzx\" (UID: \"952b6856-5488-4439-8b8f-cd2d0cf63b4c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:14.567710 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.567678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tfgt\" (UniqueName: \"kubernetes.io/projected/952b6856-5488-4439-8b8f-cd2d0cf63b4c-kube-api-access-9tfgt\") pod \"dns-operator-controller-manager-864dd78c4c-t9hzx\" (UID: \"952b6856-5488-4439-8b8f-cd2d0cf63b4c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:14.631845 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.631807 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" event={"ID":"b3e78290-588f-4795-b796-4c3443a1b817","Type":"ContainerStarted","Data":"188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c"} Jul 09 14:11:14.632054 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.631911 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:14.650459 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.650407 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" podStartSLOduration=1.240903701 podStartE2EDuration="3.650391746s" totalImagesPullingTime="2.409488045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:14.649145047 +0000 UTC m=+536.287819858" watchObservedRunningTime="2026-07-09 14:11:14.650391746 +0000 UTC m=+536.289066519" Jul 09 14:11:14.746217 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.746118 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:14.900630 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:14.900604 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx"] Jul 09 14:11:14.902118 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:14.902083 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952b6856_5488_4439_8b8f_cd2d0cf63b4c.slice/crio-016c90b4d6bbb83b973fb3230eb86838ccb4cdaae312a111071db0afba347611 WatchSource:0}: Error finding container 016c90b4d6bbb83b973fb3230eb86838ccb4cdaae312a111071db0afba347611: Status 404 returned error can't find the container with id 016c90b4d6bbb83b973fb3230eb86838ccb4cdaae312a111071db0afba347611 Jul 09 14:11:15.639109 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:15.639059 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" event={"ID":"952b6856-5488-4439-8b8f-cd2d0cf63b4c","Type":"ContainerStarted","Data":"016c90b4d6bbb83b973fb3230eb86838ccb4cdaae312a111071db0afba347611"} Jul 09 14:11:17.659709 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:17.659665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" event={"ID":"952b6856-5488-4439-8b8f-cd2d0cf63b4c","Type":"ContainerStarted","Data":"6694d7ae66e94111098cbcea4df2d338db48c850d16cec5999746fc87dcbc00b"} Jul 09 14:11:17.660059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:17.659741 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:17.696884 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:17.696822 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" podStartSLOduration=1.546908879 podStartE2EDuration="3.696805964s" totalImagesPullingTime="2.149897085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:17.689791247 +0000 UTC m=+539.328466021" watchObservedRunningTime="2026-07-09 14:11:17.696805964 +0000 UTC m=+539.335480738" Jul 09 14:11:18.839174 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.839134 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq"] Jul 09 14:11:18.843207 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.843180 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:18.846514 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.846493 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-48vw8\"" Jul 09 14:11:18.861543 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.861507 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq"] Jul 09 14:11:18.901534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.901498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56m9w\" (UniqueName: \"kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:18.901743 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:18.901546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.002439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.002404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56m9w\" (UniqueName: \"kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.002631 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.002449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.002864 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.002840 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.013775 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.013742 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56m9w\" (UniqueName: \"kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.158802 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.158708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:19.307392 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.307365 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq"] Jul 09 14:11:19.309693 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:19.309667 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf1167f_ddc6_4e48_a943_d31785e77743.slice/crio-dfacb839b8d8869a4accbad2bce1ee6e14eb24a11ce48540711c75f0fde70294 WatchSource:0}: Error finding container dfacb839b8d8869a4accbad2bce1ee6e14eb24a11ce48540711c75f0fde70294: Status 404 returned error can't find the container with id dfacb839b8d8869a4accbad2bce1ee6e14eb24a11ce48540711c75f0fde70294 Jul 09 14:11:19.672250 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:19.672195 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" event={"ID":"adf1167f-ddc6-4e48-a943-d31785e77743","Type":"ContainerStarted","Data":"dfacb839b8d8869a4accbad2bce1ee6e14eb24a11ce48540711c75f0fde70294"} Jul 09 14:11:21.610111 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:21.610077 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ss4nw" Jul 09 14:11:25.642158 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:25.642115 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:26.633890 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.633846 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-658cccdd8b-qwcvk" podUID="20315ab9-aab2-4669-bbb1-66954b679fb4" containerName="console" containerID="cri-o://378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c" gracePeriod=15 Jul 09 14:11:26.720246 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.720184 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" event={"ID":"adf1167f-ddc6-4e48-a943-d31785e77743","Type":"ContainerStarted","Data":"1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0"} Jul 09 14:11:26.720644 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.720279 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:26.745400 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.745032 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" podStartSLOduration=2.083135787 podStartE2EDuration="8.745010249s" totalImagesPullingTime="6.661874462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:26.741991427 +0000 UTC m=+548.380666200" watchObservedRunningTime="2026-07-09 14:11:26.745010249 +0000 UTC m=+548.383685024" Jul 09 14:11:26.888459 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.888393 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:11:26.980609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980568 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.980822 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980620 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.980822 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980644 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-824bv\" (UniqueName: \"kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.980822 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980675 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.980822 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980749 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.980822 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980815 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.981148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.980877 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config\") pod \"20315ab9-aab2-4669-bbb1-66954b679fb4\" (UID: \"20315ab9-aab2-4669-bbb1-66954b679fb4\") " Jul 09 14:11:26.981148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981096 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:11:26.981280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981146 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:11:26.981458 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981434 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-oauth-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:26.981562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981465 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-trusted-ca-bundle\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:26.981562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981484 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config" (OuterVolumeSpecName: "console-config") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:11:26.981688 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.981649 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca" (OuterVolumeSpecName: "service-ca") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:11:26.982983 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.982953 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:11:26.983401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.983370 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:11:26.983401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:26.983387 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv" (OuterVolumeSpecName: "kube-api-access-824bv") pod "20315ab9-aab2-4669-bbb1-66954b679fb4" (UID: "20315ab9-aab2-4669-bbb1-66954b679fb4"). InnerVolumeSpecName "kube-api-access-824bv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:11:27.082796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.082761 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-serving-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:27.082796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.082794 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-console-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:27.082796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.082808 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-824bv\" (UniqueName: \"kubernetes.io/projected/20315ab9-aab2-4669-bbb1-66954b679fb4-kube-api-access-824bv\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:27.083050 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.082817 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20315ab9-aab2-4669-bbb1-66954b679fb4-console-oauth-config\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:27.083050 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.082826 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20315ab9-aab2-4669-bbb1-66954b679fb4-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:27.728150 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.728109 2576 generic.go:356] "Generic (PLEG): container finished" podID="20315ab9-aab2-4669-bbb1-66954b679fb4" containerID="378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c" exitCode=0 Jul 09 14:11:27.728622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.728161 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658cccdd8b-qwcvk" Jul 09 14:11:27.728622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.728192 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658cccdd8b-qwcvk" event={"ID":"20315ab9-aab2-4669-bbb1-66954b679fb4","Type":"ContainerDied","Data":"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c"} Jul 09 14:11:27.728622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.728264 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658cccdd8b-qwcvk" event={"ID":"20315ab9-aab2-4669-bbb1-66954b679fb4","Type":"ContainerDied","Data":"511ab5fce7842e183f388667fd44983226dbbb742ad57ee138678d4be3ccf266"} Jul 09 14:11:27.728622 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.728293 2576 scope.go:117] "RemoveContainer" containerID="378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c" Jul 09 14:11:27.738263 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.738244 2576 scope.go:117] "RemoveContainer" containerID="378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c" Jul 09 14:11:27.738562 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:27.738538 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c\": container with ID starting with 378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c not found: ID does not exist" containerID="378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c" Jul 09 14:11:27.738610 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.738571 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c"} err="failed to get container status \"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c\": rpc error: code = NotFound desc = could not find container \"378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c\": container with ID starting with 378b0c8d279fd6919fa6b444700e6eb55866cd9a9de564a231311222d7a0654c not found: ID does not exist" Jul 09 14:11:27.752046 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.752009 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:11:27.756313 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:27.756274 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658cccdd8b-qwcvk"] Jul 09 14:11:28.667101 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.667072 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t9hzx" Jul 09 14:11:28.897218 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.897184 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20315ab9-aab2-4669-bbb1-66954b679fb4" path="/var/lib/kubelet/pods/20315ab9-aab2-4669-bbb1-66954b679fb4/volumes" Jul 09 14:11:28.897598 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.897529 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx"] Jul 09 14:11:28.897967 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.897952 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20315ab9-aab2-4669-bbb1-66954b679fb4" containerName="console" Jul 09 14:11:28.898004 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.897969 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="20315ab9-aab2-4669-bbb1-66954b679fb4" containerName="console" Jul 09 14:11:28.898057 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.898048 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="20315ab9-aab2-4669-bbb1-66954b679fb4" containerName="console" Jul 09 14:11:28.902860 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.902830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:28.904826 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.904796 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx"] Jul 09 14:11:28.905777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.905756 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 14:11:28.906010 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.905995 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7nrjf\"" Jul 09 14:11:28.906082 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:28.906016 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 14:11:29.001467 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.001362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.001628 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.001540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.001628 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.001585 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wcj\" (UniqueName: \"kubernetes.io/projected/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-kube-api-access-t6wcj\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.102532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.102488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.102532 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.102535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.102754 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.102576 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wcj\" (UniqueName: \"kubernetes.io/projected/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-kube-api-access-t6wcj\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.102754 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:29.102697 2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 14:11:29.102821 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:29.102776 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert podName:6fc54ada-51d6-47dc-a8b5-5c474a7399e9 nodeName:}" failed. No retries permitted until 2026-07-09 14:11:29.602759988 +0000 UTC m=+551.241434741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-trwbx" (UID: "6fc54ada-51d6-47dc-a8b5-5c474a7399e9") : secret "plugin-serving-cert" not found Jul 09 14:11:29.103181 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.103163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.116543 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.116516 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wcj\" (UniqueName: \"kubernetes.io/projected/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-kube-api-access-t6wcj\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.607659 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.607619 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.610039 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.610012 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fc54ada-51d6-47dc-a8b5-5c474a7399e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trwbx\" (UID: \"6fc54ada-51d6-47dc-a8b5-5c474a7399e9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.813462 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.813410 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" Jul 09 14:11:29.953707 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:29.953672 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx"] Jul 09 14:11:29.955357 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:29.955330 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fc54ada_51d6_47dc_a8b5_5c474a7399e9.slice/crio-f950c0b3876a8cc2085e283620f1e3102407414d7e83f09ffa7b021b87a6c942 WatchSource:0}: Error finding container f950c0b3876a8cc2085e283620f1e3102407414d7e83f09ffa7b021b87a6c942: Status 404 returned error can't find the container with id f950c0b3876a8cc2085e283620f1e3102407414d7e83f09ffa7b021b87a6c942 Jul 09 14:11:30.750679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:30.750642 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" event={"ID":"6fc54ada-51d6-47dc-a8b5-5c474a7399e9","Type":"ContainerStarted","Data":"f950c0b3876a8cc2085e283620f1e3102407414d7e83f09ffa7b021b87a6c942"} Jul 09 14:11:37.730802 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:37.730765 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.518857 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.518814 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq"] Jul 09 14:11:39.519360 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.519052 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" containerName="manager" containerID="cri-o://1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0" gracePeriod=2 Jul 09 14:11:39.525750 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.525719 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq"] Jul 09 14:11:39.533697 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.533650 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:11:39.534573 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.534547 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" containerName="manager" Jul 09 14:11:39.534729 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.534716 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" containerName="manager" Jul 09 14:11:39.535013 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.534992 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" containerName="manager" Jul 09 14:11:39.538833 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.538812 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj"] Jul 09 14:11:39.539073 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.539050 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.539295 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.539260 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" podUID="b3e78290-588f-4795-b796-4c3443a1b817" containerName="manager" containerID="cri-o://188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c" gracePeriod=2 Jul 09 14:11:39.544974 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.544784 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.547317 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.546747 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj"] Jul 09 14:11:39.555014 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.554958 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:11:39.570614 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.570207 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw"] Jul 09 14:11:39.570910 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.570886 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3e78290-588f-4795-b796-4c3443a1b817" containerName="manager" Jul 09 14:11:39.571019 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.570912 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3e78290-588f-4795-b796-4c3443a1b817" containerName="manager" Jul 09 14:11:39.571075 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.571035 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3e78290-588f-4795-b796-4c3443a1b817" containerName="manager" Jul 09 14:11:39.574862 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.574838 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:39.580772 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.580712 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.583426 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.583298 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.585737 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.585698 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.586864 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.586833 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw"] Jul 09 14:11:39.620170 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.620125 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr7p7\" (UniqueName: \"kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.620375 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.620258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.720906 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.720869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.721352 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.721015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxr9k\" (UniqueName: \"kubernetes.io/projected/eb315145-8661-44a4-921d-be7a510b6b82-kube-api-access-rxr9k\") pod \"limitador-operator-controller-manager-859d7b78bd-bnqcw\" (UID: \"eb315145-8661-44a4-921d-be7a510b6b82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:39.721352 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.721075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr7p7\" (UniqueName: \"kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.721352 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.721322 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.730713 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.730689 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr7p7\" (UniqueName: \"kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k9plb\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.809301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.809271 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.811734 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.811713 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.811987 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.811965 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.814746 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.814725 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.817104 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.817083 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.819413 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.819387 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.821971 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.821945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxr9k\" (UniqueName: \"kubernetes.io/projected/eb315145-8661-44a4-921d-be7a510b6b82-kube-api-access-rxr9k\") pod \"limitador-operator-controller-manager-859d7b78bd-bnqcw\" (UID: \"eb315145-8661-44a4-921d-be7a510b6b82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:39.823753 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.823732 2576 generic.go:356] "Generic (PLEG): container finished" podID="adf1167f-ddc6-4e48-a943-d31785e77743" containerID="1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0" exitCode=0 Jul 09 14:11:39.823871 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.823778 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.823871 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.823821 2576 scope.go:117] "RemoveContainer" containerID="1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0" Jul 09 14:11:39.825252 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.825206 2576 generic.go:356] "Generic (PLEG): container finished" podID="b3e78290-588f-4795-b796-4c3443a1b817" containerID="188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c" exitCode=0 Jul 09 14:11:39.825344 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.825327 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.826620 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.826595 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.831315 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.831290 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.831825 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.831804 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxr9k\" (UniqueName: \"kubernetes.io/projected/eb315145-8661-44a4-921d-be7a510b6b82-kube-api-access-rxr9k\") pod \"limitador-operator-controller-manager-859d7b78bd-bnqcw\" (UID: \"eb315145-8661-44a4-921d-be7a510b6b82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:39.833764 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.833725 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:39.835826 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.835809 2576 scope.go:117] "RemoveContainer" containerID="1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0" Jul 09 14:11:39.836052 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.836023 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:39.836185 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.836161 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0\": container with ID starting with 1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0 not found: ID does not exist" containerID="1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0" Jul 09 14:11:39.836252 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.836200 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0"} err="failed to get container status \"1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0\": rpc error: code = NotFound desc = could not find container \"1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0\": container with ID starting with 1dea46d123fd029aa535353470b7c8d8e38f4ce9ad5874a98caf36c12b1e48f0 not found: ID does not exist" Jul 09 14:11:39.836252 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.836222 2576 scope.go:117] "RemoveContainer" containerID="188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c" Jul 09 14:11:39.846922 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.846897 2576 scope.go:117] "RemoveContainer" containerID="188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c" Jul 09 14:11:39.847266 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:39.847215 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c\": container with ID starting with 188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c not found: ID does not exist" containerID="188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c" Jul 09 14:11:39.847372 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.847270 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c"} err="failed to get container status \"188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c\": rpc error: code = NotFound desc = could not find container \"188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c\": container with ID starting with 188f537799d0bcb2ba7cb0ff63975c8d78382dcf53bc3a6fb0833cc6b058066c not found: ID does not exist" Jul 09 14:11:39.923444 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.923404 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2bw5\" (UniqueName: \"kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5\") pod \"b3e78290-588f-4795-b796-4c3443a1b817\" (UID: \"b3e78290-588f-4795-b796-4c3443a1b817\") " Jul 09 14:11:39.923636 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.923462 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume\") pod \"adf1167f-ddc6-4e48-a943-d31785e77743\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " Jul 09 14:11:39.923636 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.923506 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56m9w\" (UniqueName: \"kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w\") pod \"adf1167f-ddc6-4e48-a943-d31785e77743\" (UID: \"adf1167f-ddc6-4e48-a943-d31785e77743\") " Jul 09 14:11:39.924116 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.924089 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "adf1167f-ddc6-4e48-a943-d31785e77743" (UID: "adf1167f-ddc6-4e48-a943-d31785e77743"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:11:39.926219 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.926194 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w" (OuterVolumeSpecName: "kube-api-access-56m9w") pod "adf1167f-ddc6-4e48-a943-d31785e77743" (UID: "adf1167f-ddc6-4e48-a943-d31785e77743"). InnerVolumeSpecName "kube-api-access-56m9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:11:39.926362 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.926255 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5" (OuterVolumeSpecName: "kube-api-access-s2bw5") pod "b3e78290-588f-4795-b796-4c3443a1b817" (UID: "b3e78290-588f-4795-b796-4c3443a1b817"). InnerVolumeSpecName "kube-api-access-s2bw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:11:39.956685 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.956648 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:39.967384 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:39.967356 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:40.024798 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.024767 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2bw5\" (UniqueName: \"kubernetes.io/projected/b3e78290-588f-4795-b796-4c3443a1b817-kube-api-access-s2bw5\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:40.024798 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.024797 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adf1167f-ddc6-4e48-a943-d31785e77743-extensions-socket-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:40.024983 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.024808 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56m9w\" (UniqueName: \"kubernetes.io/projected/adf1167f-ddc6-4e48-a943-d31785e77743-kube-api-access-56m9w\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:11:40.149106 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.149063 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:40.151732 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.151696 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:40.156409 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.156366 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:40.159117 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.159083 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:40.159581 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.159561 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:11:40.160670 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:40.160638 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea12ac87_b5f9_4f75_8c05_00b8b516fe03.slice/crio-09ccc5c70a508a2f37f01074305d92a327879427a6eb9e97338e647617bde7bb WatchSource:0}: Error finding container 09ccc5c70a508a2f37f01074305d92a327879427a6eb9e97338e647617bde7bb: Status 404 returned error can't find the container with id 09ccc5c70a508a2f37f01074305d92a327879427a6eb9e97338e647617bde7bb Jul 09 14:11:40.171114 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.171089 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw"] Jul 09 14:11:40.173058 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:11:40.173035 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb315145_8661_44a4_921d_be7a510b6b82.slice/crio-772518b666f623ba86713522daaf8390eae7b64c7674480f0e92b05078d4aaa4 WatchSource:0}: Error finding container 772518b666f623ba86713522daaf8390eae7b64c7674480f0e92b05078d4aaa4: Status 404 returned error can't find the container with id 772518b666f623ba86713522daaf8390eae7b64c7674480f0e92b05078d4aaa4 Jul 09 14:11:40.835865 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.835815 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" event={"ID":"ea12ac87-b5f9-4f75-8c05-00b8b516fe03","Type":"ContainerStarted","Data":"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94"} Jul 09 14:11:40.836325 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.835911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" event={"ID":"ea12ac87-b5f9-4f75-8c05-00b8b516fe03","Type":"ContainerStarted","Data":"09ccc5c70a508a2f37f01074305d92a327879427a6eb9e97338e647617bde7bb"} Jul 09 14:11:40.836325 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.835950 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:40.838868 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.838836 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" event={"ID":"eb315145-8661-44a4-921d-be7a510b6b82","Type":"ContainerStarted","Data":"d0fd92cb60bf7a23d40f7e322a58a1e67f8e6f027df878508ad8924b4d395e8b"} Jul 09 14:11:40.838868 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.838849 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:40.839064 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.838890 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" event={"ID":"eb315145-8661-44a4-921d-be7a510b6b82","Type":"ContainerStarted","Data":"772518b666f623ba86713522daaf8390eae7b64c7674480f0e92b05078d4aaa4"} Jul 09 14:11:40.839064 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.838936 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:40.861781 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.861718 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" podStartSLOduration=1.861698878 podStartE2EDuration="1.861698878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:40.861307316 +0000 UTC m=+562.499982113" watchObservedRunningTime="2026-07-09 14:11:40.861698878 +0000 UTC m=+562.500373651" Jul 09 14:11:40.863551 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.863519 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:40.865703 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.865669 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ffjxq\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ffjxq" Jul 09 14:11:40.890790 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:11:40.890751 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4h7nj\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="b3e78290-588f-4795-b796-4c3443a1b817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4h7nj" Jul 09 14:11:40.895305 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.895273 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adf1167f-ddc6-4e48-a943-d31785e77743" path="/var/lib/kubelet/pods/adf1167f-ddc6-4e48-a943-d31785e77743/volumes" Jul 09 14:11:40.895697 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:40.895678 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3e78290-588f-4795-b796-4c3443a1b817" path="/var/lib/kubelet/pods/b3e78290-588f-4795-b796-4c3443a1b817/volumes" Jul 09 14:11:51.848947 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:51.848910 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" Jul 09 14:11:51.849440 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:51.849017 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:11:51.870012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:51.869378 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bnqcw" podStartSLOduration=12.869358146 podStartE2EDuration="12.869358146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:40.887604516 +0000 UTC m=+562.526279290" watchObservedRunningTime="2026-07-09 14:11:51.869358146 +0000 UTC m=+573.508032920" Jul 09 14:11:56.965376 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:56.965340 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" event={"ID":"6fc54ada-51d6-47dc-a8b5-5c474a7399e9","Type":"ContainerStarted","Data":"b6d13764d2ecdfd6ffc34ffa166e2be56ed83a5978e20a565c4ba0477a2cd3e3"} Jul 09 14:11:56.985272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:11:56.985202 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trwbx" podStartSLOduration=2.813078532 podStartE2EDuration="28.985184227s" totalImagesPullingTime="26.172105695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:56.982925934 +0000 UTC m=+578.621600912" watchObservedRunningTime="2026-07-09 14:11:56.985184227 +0000 UTC m=+578.623859000" Jul 09 14:12:04.032991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.032949 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:12:04.033463 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.033182 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" podUID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" containerName="manager" containerID="cri-o://24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94" gracePeriod=10 Jul 09 14:12:04.291205 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.291130 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:12:04.356361 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.356321 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume\") pod \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " Jul 09 14:12:04.356536 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.356415 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr7p7\" (UniqueName: \"kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7\") pod \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\" (UID: \"ea12ac87-b5f9-4f75-8c05-00b8b516fe03\") " Jul 09 14:12:04.356777 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.356752 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ea12ac87-b5f9-4f75-8c05-00b8b516fe03" (UID: "ea12ac87-b5f9-4f75-8c05-00b8b516fe03"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:12:04.358587 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.358556 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7" (OuterVolumeSpecName: "kube-api-access-cr7p7") pod "ea12ac87-b5f9-4f75-8c05-00b8b516fe03" (UID: "ea12ac87-b5f9-4f75-8c05-00b8b516fe03"). InnerVolumeSpecName "kube-api-access-cr7p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:12:04.457757 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.457708 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr7p7\" (UniqueName: \"kubernetes.io/projected/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-kube-api-access-cr7p7\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:12:04.457757 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:04.457747 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea12ac87-b5f9-4f75-8c05-00b8b516fe03-extensions-socket-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:12:05.015689 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.015655 2576 generic.go:356] "Generic (PLEG): container finished" podID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" containerID="24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94" exitCode=0 Jul 09 14:12:05.015689 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.015694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" event={"ID":"ea12ac87-b5f9-4f75-8c05-00b8b516fe03","Type":"ContainerDied","Data":"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94"} Jul 09 14:12:05.015953 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.015704 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" Jul 09 14:12:05.015953 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.015737 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb" event={"ID":"ea12ac87-b5f9-4f75-8c05-00b8b516fe03","Type":"ContainerDied","Data":"09ccc5c70a508a2f37f01074305d92a327879427a6eb9e97338e647617bde7bb"} Jul 09 14:12:05.015953 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.015758 2576 scope.go:117] "RemoveContainer" containerID="24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94" Jul 09 14:12:05.025428 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.025407 2576 scope.go:117] "RemoveContainer" containerID="24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94" Jul 09 14:12:05.025732 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:12:05.025713 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94\": container with ID starting with 24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94 not found: ID does not exist" containerID="24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94" Jul 09 14:12:05.025790 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.025741 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94"} err="failed to get container status \"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94\": rpc error: code = NotFound desc = could not find container \"24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94\": container with ID starting with 24761b0bd5639f46bc0c54170620b162fba4196673856a94f4234f47ebaa8a94 not found: ID does not exist" Jul 09 14:12:05.037649 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.037615 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:12:05.041317 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:05.041283 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k9plb"] Jul 09 14:12:06.895335 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:06.895298 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" path="/var/lib/kubelet/pods/ea12ac87-b5f9-4f75-8c05-00b8b516fe03/volumes" Jul 09 14:12:08.280322 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.280278 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67"] Jul 09 14:12:08.280799 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.280733 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" containerName="manager" Jul 09 14:12:08.280799 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.280744 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" containerName="manager" Jul 09 14:12:08.280926 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.280809 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea12ac87-b5f9-4f75-8c05-00b8b516fe03" containerName="manager" Jul 09 14:12:08.299712 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.299677 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67"] Jul 09 14:12:08.299904 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.299787 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.302419 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.302391 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 14:12:08.302601 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.302431 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6jm5t\"" Jul 09 14:12:08.397832 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.397786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398037 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.397842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398037 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.397940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d863e139-fe8f-44cd-b968-cf8c45a74d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398037 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.397969 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2fvw\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-kube-api-access-v2fvw\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398037 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.398018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.398039 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.398063 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.398116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.398307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.398185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498793 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498806 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d863e139-fe8f-44cd-b968-cf8c45a74d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2fvw\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-kube-api-access-v2fvw\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.498974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498907 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.498997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499032 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499067 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499223 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499503 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499336 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499503 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499450 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499609 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499570 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.499668 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.499628 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d863e139-fe8f-44cd-b968-cf8c45a74d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.501148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.501125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.501493 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.501471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.508274 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.508218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.508412 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.508338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2fvw\" (UniqueName: \"kubernetes.io/projected/d863e139-fe8f-44cd-b968-cf8c45a74d57-kube-api-access-v2fvw\") pod \"maas-default-gateway-openshift-default-56654dcf7-2rp67\" (UID: \"d863e139-fe8f-44cd-b968-cf8c45a74d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.613122 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.613076 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:08.762786 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:08.762746 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67"] Jul 09 14:12:08.764535 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:12:08.764502 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd863e139_fe8f_44cd_b968_cf8c45a74d57.slice/crio-14b1e418a4e79ac646ea953b0d056177d7494d56ec9465d04c46661abfc187a9 WatchSource:0}: Error finding container 14b1e418a4e79ac646ea953b0d056177d7494d56ec9465d04c46661abfc187a9: Status 404 returned error can't find the container with id 14b1e418a4e79ac646ea953b0d056177d7494d56ec9465d04c46661abfc187a9 Jul 09 14:12:09.042291 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:09.042190 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" event={"ID":"d863e139-fe8f-44cd-b968-cf8c45a74d57","Type":"ContainerStarted","Data":"14b1e418a4e79ac646ea953b0d056177d7494d56ec9465d04c46661abfc187a9"} Jul 09 14:12:11.648661 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:11.648616 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:12:11.649042 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:11.648697 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:12:11.649042 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:11.648727 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:12:12.063585 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:12.063551 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" event={"ID":"d863e139-fe8f-44cd-b968-cf8c45a74d57","Type":"ContainerStarted","Data":"7e4dceec9c562e9d755ca4bca534d68a9f96d692567c53ac1de2baae1eac1d6c"} Jul 09 14:12:12.087563 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:12.087509 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" podStartSLOduration=1.205991897 podStartE2EDuration="4.087494756s" totalImagesPullingTime="2.881502859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:12:12.085083748 +0000 UTC m=+593.723758548" watchObservedRunningTime="2026-07-09 14:12:12.087494756 +0000 UTC m=+593.726169530" Jul 09 14:12:12.614211 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:12.614173 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:12.619687 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:12.619660 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:13.069379 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:13.069344 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:13.070498 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:13.070472 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:12:18.842171 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:18.842143 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:12:18.842649 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:18.842196 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:12:18.845912 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:18.845888 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:12:18.846047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:18.845944 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:12:24.809772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.809733 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:24.862307 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.862270 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:24.862451 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.862278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:24.864923 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.864899 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 14:12:24.909093 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.909057 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:24.963505 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.963460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96d64\" (UniqueName: \"kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:24.963658 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:24.963545 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.064623 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.064529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96d64\" (UniqueName: \"kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.064623 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.064602 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.065276 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.065226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.072631 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.072607 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96d64\" (UniqueName: \"kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64\") pod \"limitador-limitador-64c94fd64f-tmvcm\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.175359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.175319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:25.365639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.365610 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:25.368218 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:12:25.368191 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bb3c4a_0512_4805_8794_77f059c9679d.slice/crio-45172ed24326158cbf47875600ebabb6f15a944001ed958418deff0f2239e81c WatchSource:0}: Error finding container 45172ed24326158cbf47875600ebabb6f15a944001ed958418deff0f2239e81c: Status 404 returned error can't find the container with id 45172ed24326158cbf47875600ebabb6f15a944001ed958418deff0f2239e81c Jul 09 14:12:25.796807 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.796729 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:12:25.856219 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.856180 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:12:25.856692 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.856316 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:25.859665 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.859631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xm6t4\"" Jul 09 14:12:25.972035 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:25.971992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9j2l\" (UniqueName: \"kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l\") pod \"authorino-75d556bd77-wwn6j\" (UID: \"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af\") " pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:26.073152 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:26.073112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9j2l\" (UniqueName: \"kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l\") pod \"authorino-75d556bd77-wwn6j\" (UID: \"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af\") " pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:26.082249 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:26.082191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9j2l\" (UniqueName: \"kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l\") pod \"authorino-75d556bd77-wwn6j\" (UID: \"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af\") " pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:26.154758 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:26.154717 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" event={"ID":"94bb3c4a-0512-4805-8794-77f059c9679d","Type":"ContainerStarted","Data":"45172ed24326158cbf47875600ebabb6f15a944001ed958418deff0f2239e81c"} Jul 09 14:12:26.168032 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:26.167997 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:26.359117 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:26.359043 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:12:26.359926 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:12:26.359896 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8a4304e_9d8e_45d4_8afc_b5c3bdca18af.slice/crio-592ddf74f5563028210bc7b6f95be06be5f2ae8ff3aafaba0062f7c7bcd5346c WatchSource:0}: Error finding container 592ddf74f5563028210bc7b6f95be06be5f2ae8ff3aafaba0062f7c7bcd5346c: Status 404 returned error can't find the container with id 592ddf74f5563028210bc7b6f95be06be5f2ae8ff3aafaba0062f7c7bcd5346c Jul 09 14:12:27.168414 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:27.168367 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwn6j" event={"ID":"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af","Type":"ContainerStarted","Data":"592ddf74f5563028210bc7b6f95be06be5f2ae8ff3aafaba0062f7c7bcd5346c"} Jul 09 14:12:31.198421 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:31.198382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwn6j" event={"ID":"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af","Type":"ContainerStarted","Data":"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe"} Jul 09 14:12:31.199859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:31.199829 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" event={"ID":"94bb3c4a-0512-4805-8794-77f059c9679d","Type":"ContainerStarted","Data":"740fdee50c249b345dac532467672968958598444513141b04608fa328db9794"} Jul 09 14:12:31.199994 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:31.199963 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:31.217022 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:31.216974 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wwn6j" podStartSLOduration=2.4038139210000002 podStartE2EDuration="6.216958677s" totalImagesPullingTime="3.813144756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:12:31.214100115 +0000 UTC m=+612.852774889" watchObservedRunningTime="2026-07-09 14:12:31.216958677 +0000 UTC m=+612.855633430" Jul 09 14:12:39.741217 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:39.741156 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" podStartSLOduration=10.950132064 podStartE2EDuration="15.74113627s" totalImagesPullingTime="4.791004206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:12:31.232059955 +0000 UTC m=+612.870734730" watchObservedRunningTime="2026-07-09 14:12:39.74113627 +0000 UTC m=+621.379811043" Jul 09 14:12:39.743210 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:39.743188 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:39.743473 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:39.743444 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" podUID="94bb3c4a-0512-4805-8794-77f059c9679d" containerName="limitador" containerID="cri-o://740fdee50c249b345dac532467672968958598444513141b04608fa328db9794" gracePeriod=30 Jul 09 14:12:39.744538 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:39.744512 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:40.256255 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.256207 2576 generic.go:356] "Generic (PLEG): container finished" podID="94bb3c4a-0512-4805-8794-77f059c9679d" containerID="740fdee50c249b345dac532467672968958598444513141b04608fa328db9794" exitCode=0 Jul 09 14:12:40.256445 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.256262 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" event={"ID":"94bb3c4a-0512-4805-8794-77f059c9679d","Type":"ContainerDied","Data":"740fdee50c249b345dac532467672968958598444513141b04608fa328db9794"} Jul 09 14:12:40.300927 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.300897 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:40.409106 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.409073 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96d64\" (UniqueName: \"kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64\") pod \"94bb3c4a-0512-4805-8794-77f059c9679d\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " Jul 09 14:12:40.409283 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.409225 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file\") pod \"94bb3c4a-0512-4805-8794-77f059c9679d\" (UID: \"94bb3c4a-0512-4805-8794-77f059c9679d\") " Jul 09 14:12:40.409534 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.409505 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file" (OuterVolumeSpecName: "config-file") pod "94bb3c4a-0512-4805-8794-77f059c9679d" (UID: "94bb3c4a-0512-4805-8794-77f059c9679d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:12:40.411136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.411110 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64" (OuterVolumeSpecName: "kube-api-access-96d64") pod "94bb3c4a-0512-4805-8794-77f059c9679d" (UID: "94bb3c4a-0512-4805-8794-77f059c9679d"). InnerVolumeSpecName "kube-api-access-96d64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:12:40.510442 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.510404 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96d64\" (UniqueName: \"kubernetes.io/projected/94bb3c4a-0512-4805-8794-77f059c9679d-kube-api-access-96d64\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:12:40.510442 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:40.510438 2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94bb3c4a-0512-4805-8794-77f059c9679d-config-file\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:12:41.263607 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:41.263518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" event={"ID":"94bb3c4a-0512-4805-8794-77f059c9679d","Type":"ContainerDied","Data":"45172ed24326158cbf47875600ebabb6f15a944001ed958418deff0f2239e81c"} Jul 09 14:12:41.263607 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:41.263582 2576 scope.go:117] "RemoveContainer" containerID="740fdee50c249b345dac532467672968958598444513141b04608fa328db9794" Jul 09 14:12:41.264073 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:41.263536 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tmvcm" Jul 09 14:12:41.282407 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:41.282376 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:41.286536 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:41.286510 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tmvcm"] Jul 09 14:12:42.895775 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:42.895738 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94bb3c4a-0512-4805-8794-77f059c9679d" path="/var/lib/kubelet/pods/94bb3c4a-0512-4805-8794-77f059c9679d/volumes" Jul 09 14:12:45.132382 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.132347 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw"] Jul 09 14:12:45.132989 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.132967 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94bb3c4a-0512-4805-8794-77f059c9679d" containerName="limitador" Jul 09 14:12:45.133078 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.132992 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="94bb3c4a-0512-4805-8794-77f059c9679d" containerName="limitador" Jul 09 14:12:45.133134 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.133123 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="94bb3c4a-0512-4805-8794-77f059c9679d" containerName="limitador" Jul 09 14:12:45.138383 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.138359 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.141247 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.141211 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 14:12:45.142666 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.142645 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 14:12:45.142794 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.142734 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-msgkl\"" Jul 09 14:12:45.142863 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.142694 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 14:12:45.146403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.146096 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw"] Jul 09 14:12:45.256052 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.256009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b684a681-7cf3-4cca-94e3-627306e2f57a-data\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.256283 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.256127 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rklrx\" (UniqueName: \"kubernetes.io/projected/b684a681-7cf3-4cca-94e3-627306e2f57a-kube-api-access-rklrx\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.357190 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.357154 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rklrx\" (UniqueName: \"kubernetes.io/projected/b684a681-7cf3-4cca-94e3-627306e2f57a-kube-api-access-rklrx\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.357411 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.357277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b684a681-7cf3-4cca-94e3-627306e2f57a-data\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.357674 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.357653 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b684a681-7cf3-4cca-94e3-627306e2f57a-data\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.366737 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.366707 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rklrx\" (UniqueName: \"kubernetes.io/projected/b684a681-7cf3-4cca-94e3-627306e2f57a-kube-api-access-rklrx\") pod \"postgres-69fb4594cf-6pgfw\" (UID: \"b684a681-7cf3-4cca-94e3-627306e2f57a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.450958 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.450853 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:45.616146 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.616117 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw"] Jul 09 14:12:45.618285 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:12:45.618221 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb684a681_7cf3_4cca_94e3_627306e2f57a.slice/crio-b6c57a0756f8367d33c8a8c98db3ae81390d50673a695c0a5a71e7d54dfa9b2f WatchSource:0}: Error finding container b6c57a0756f8367d33c8a8c98db3ae81390d50673a695c0a5a71e7d54dfa9b2f: Status 404 returned error can't find the container with id b6c57a0756f8367d33c8a8c98db3ae81390d50673a695c0a5a71e7d54dfa9b2f Jul 09 14:12:45.619598 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:45.619578 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:12:46.295548 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:46.295506 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" event={"ID":"b684a681-7cf3-4cca-94e3-627306e2f57a","Type":"ContainerStarted","Data":"b6c57a0756f8367d33c8a8c98db3ae81390d50673a695c0a5a71e7d54dfa9b2f"} Jul 09 14:12:51.332268 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:51.332212 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" event={"ID":"b684a681-7cf3-4cca-94e3-627306e2f57a","Type":"ContainerStarted","Data":"9f263a6b9641bb56b19ab8ef9506acc0bda43a20e731b2896c5cda9adb95b568"} Jul 09 14:12:51.332741 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:51.332315 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:51.351494 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:51.351435 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" podStartSLOduration=1.063497398 podStartE2EDuration="6.351418036s" totalImagesPullingTime="5.287920638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:12:51.349673955 +0000 UTC m=+632.988348775" watchObservedRunningTime="2026-07-09 14:12:51.351418036 +0000 UTC m=+632.990092809" Jul 09 14:12:57.369771 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:57.369737 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6pgfw" Jul 09 14:12:59.304905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.304859 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:12:59.305409 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.305092 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wwn6j" podUID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" containerName="authorino" containerID="cri-o://a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe" gracePeriod=30 Jul 09 14:12:59.616960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.616918 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:12:59.707627 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.707587 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9j2l\" (UniqueName: \"kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l\") pod \"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af\" (UID: \"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af\") " Jul 09 14:12:59.709667 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.709644 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l" (OuterVolumeSpecName: "kube-api-access-l9j2l") pod "d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" (UID: "d8a4304e-9d8e-45d4-8afc-b5c3bdca18af"). InnerVolumeSpecName "kube-api-access-l9j2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:12:59.808978 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:12:59.808931 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9j2l\" (UniqueName: \"kubernetes.io/projected/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af-kube-api-access-l9j2l\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:13:00.391148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.391110 2576 generic.go:356] "Generic (PLEG): container finished" podID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" containerID="a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe" exitCode=0 Jul 09 14:13:00.391556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.391152 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwn6j" Jul 09 14:13:00.391556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.391190 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwn6j" event={"ID":"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af","Type":"ContainerDied","Data":"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe"} Jul 09 14:13:00.391556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.391259 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwn6j" event={"ID":"d8a4304e-9d8e-45d4-8afc-b5c3bdca18af","Type":"ContainerDied","Data":"592ddf74f5563028210bc7b6f95be06be5f2ae8ff3aafaba0062f7c7bcd5346c"} Jul 09 14:13:00.391556 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.391285 2576 scope.go:117] "RemoveContainer" containerID="a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe" Jul 09 14:13:00.401860 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.401588 2576 scope.go:117] "RemoveContainer" containerID="a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe" Jul 09 14:13:00.402121 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:13:00.402054 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe\": container with ID starting with a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe not found: ID does not exist" containerID="a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe" Jul 09 14:13:00.402121 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.402089 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe"} err="failed to get container status \"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe\": rpc error: code = NotFound desc = could not find container \"a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe\": container with ID starting with a2cffb7962f32ed9447d2f321971f39d15aeb7582b94746d60e65a1bccb99cfe not found: ID does not exist" Jul 09 14:13:00.414193 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.414157 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:13:00.425181 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.418506 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwn6j"] Jul 09 14:13:00.896618 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:00.896581 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" path="/var/lib/kubelet/pods/d8a4304e-9d8e-45d4-8afc-b5c3bdca18af/volumes" Jul 09 14:13:03.616183 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.616140 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:03.616828 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.616805 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" containerName="authorino" Jul 09 14:13:03.616918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.616830 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" containerName="authorino" Jul 09 14:13:03.616974 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.616966 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a4304e-9d8e-45d4-8afc-b5c3bdca18af" containerName="authorino" Jul 09 14:13:03.690581 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.690538 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:03.690771 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.690671 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.693924 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.693880 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 14:13:03.694068 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.693905 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 14:13:03.694068 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.693904 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q5klz\"" Jul 09 14:13:03.790734 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.790699 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:13:03.840986 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.840954 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:13:03.841154 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.841067 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:03.850077 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.850049 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2drn\" (UniqueName: \"kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.850198 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.850101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.951324 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.951204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.951479 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.951330 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5tf\" (UniqueName: \"kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:03.951479 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.951392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:03.951479 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.951456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2drn\" (UniqueName: \"kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.953799 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.953778 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:03.959842 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:03.959809 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2drn\" (UniqueName: \"kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn\") pod \"maas-controller-84c65cfbf9-87nl9\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:04.001369 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.001331 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:04.052516 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.052483 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5tf\" (UniqueName: \"kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:04.052688 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.052548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:04.055072 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.055044 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:04.061810 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.061785 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5tf\" (UniqueName: \"kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf\") pod \"maas-controller-65dbd89dd4-f9wgx\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:04.151696 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.151655 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:04.189691 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.189665 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:04.191007 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:13:04.190974 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e05343_e0bb_4813_bebf_f2319a90329d.slice/crio-af70ea85636b29729cf0a595d34fa7cb332620b7e50b817770fafec422798f62 WatchSource:0}: Error finding container af70ea85636b29729cf0a595d34fa7cb332620b7e50b817770fafec422798f62: Status 404 returned error can't find the container with id af70ea85636b29729cf0a595d34fa7cb332620b7e50b817770fafec422798f62 Jul 09 14:13:04.329360 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.329333 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:13:04.330650 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:13:04.330622 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc537bea8_e746_41cd_b573_3e4f98cefd67.slice/crio-185207c91a51c7dc43303ef0b83bcd56fe956446cd6448309739c4794c14f70b WatchSource:0}: Error finding container 185207c91a51c7dc43303ef0b83bcd56fe956446cd6448309739c4794c14f70b: Status 404 returned error can't find the container with id 185207c91a51c7dc43303ef0b83bcd56fe956446cd6448309739c4794c14f70b Jul 09 14:13:04.416183 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.416145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" event={"ID":"c537bea8-e746-41cd-b573-3e4f98cefd67","Type":"ContainerStarted","Data":"185207c91a51c7dc43303ef0b83bcd56fe956446cd6448309739c4794c14f70b"} Jul 09 14:13:04.417401 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:04.417371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" event={"ID":"a7e05343-e0bb-4813-bebf-f2319a90329d","Type":"ContainerStarted","Data":"af70ea85636b29729cf0a595d34fa7cb332620b7e50b817770fafec422798f62"} Jul 09 14:13:07.439995 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.439952 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" event={"ID":"c537bea8-e746-41cd-b573-3e4f98cefd67","Type":"ContainerStarted","Data":"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc"} Jul 09 14:13:07.440507 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.440051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:07.441612 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.441585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" event={"ID":"a7e05343-e0bb-4813-bebf-f2319a90329d","Type":"ContainerStarted","Data":"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e"} Jul 09 14:13:07.441734 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.441663 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:07.460455 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.460397 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" podStartSLOduration=2.4018550530000002 podStartE2EDuration="4.460380199s" totalImagesPullingTime="2.058525146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:13:07.457958413 +0000 UTC m=+649.096633214" watchObservedRunningTime="2026-07-09 14:13:07.460380199 +0000 UTC m=+649.099054972" Jul 09 14:13:07.479458 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:07.479395 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" podStartSLOduration=2.273837175 podStartE2EDuration="4.479374418s" totalImagesPullingTime="2.205537243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:13:07.47673129 +0000 UTC m=+649.115406064" watchObservedRunningTime="2026-07-09 14:13:07.479374418 +0000 UTC m=+649.118049194" Jul 09 14:13:18.450393 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.450357 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:18.450929 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.450445 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:13:18.497887 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.497847 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:18.523369 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.522693 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" podUID="a7e05343-e0bb-4813-bebf-f2319a90329d" containerName="manager" containerID="cri-o://287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e" gracePeriod=30 Jul 09 14:13:18.785010 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.784979 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:18.886247 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.886210 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert\") pod \"a7e05343-e0bb-4813-bebf-f2319a90329d\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " Jul 09 14:13:18.886433 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.886323 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2drn\" (UniqueName: \"kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn\") pod \"a7e05343-e0bb-4813-bebf-f2319a90329d\" (UID: \"a7e05343-e0bb-4813-bebf-f2319a90329d\") " Jul 09 14:13:18.888439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.888404 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a7e05343-e0bb-4813-bebf-f2319a90329d" (UID: "a7e05343-e0bb-4813-bebf-f2319a90329d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:13:18.888439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.888420 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn" (OuterVolumeSpecName: "kube-api-access-l2drn") pod "a7e05343-e0bb-4813-bebf-f2319a90329d" (UID: "a7e05343-e0bb-4813-bebf-f2319a90329d"). InnerVolumeSpecName "kube-api-access-l2drn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:13:18.988221 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.988183 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2drn\" (UniqueName: \"kubernetes.io/projected/a7e05343-e0bb-4813-bebf-f2319a90329d-kube-api-access-l2drn\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:13:18.988221 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:18.988217 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e05343-e0bb-4813-bebf-f2319a90329d-webhook-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:13:19.529572 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.529535 2576 generic.go:356] "Generic (PLEG): container finished" podID="a7e05343-e0bb-4813-bebf-f2319a90329d" containerID="287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e" exitCode=0 Jul 09 14:13:19.530059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.529579 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" Jul 09 14:13:19.530059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.529586 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" event={"ID":"a7e05343-e0bb-4813-bebf-f2319a90329d","Type":"ContainerDied","Data":"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e"} Jul 09 14:13:19.530059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.529698 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84c65cfbf9-87nl9" event={"ID":"a7e05343-e0bb-4813-bebf-f2319a90329d","Type":"ContainerDied","Data":"af70ea85636b29729cf0a595d34fa7cb332620b7e50b817770fafec422798f62"} Jul 09 14:13:19.530059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.529721 2576 scope.go:117] "RemoveContainer" containerID="287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e" Jul 09 14:13:19.539184 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.539164 2576 scope.go:117] "RemoveContainer" containerID="287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e" Jul 09 14:13:19.539506 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:13:19.539481 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e\": container with ID starting with 287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e not found: ID does not exist" containerID="287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e" Jul 09 14:13:19.539566 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.539508 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e"} err="failed to get container status \"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e\": rpc error: code = NotFound desc = could not find container \"287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e\": container with ID starting with 287f66c17090ba8dce60cbb6df1191cd1bf212b6ee09a3833ac920b7bce6416e not found: ID does not exist" Jul 09 14:13:19.548014 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.547982 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:19.553805 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:19.553776 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84c65cfbf9-87nl9"] Jul 09 14:13:20.895754 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:20.895719 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e05343-e0bb-4813-bebf-f2319a90329d" path="/var/lib/kubelet/pods/a7e05343-e0bb-4813-bebf-f2319a90329d/volumes" Jul 09 14:13:39.607820 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.607779 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l"] Jul 09 14:13:39.608302 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.608280 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e05343-e0bb-4813-bebf-f2319a90329d" containerName="manager" Jul 09 14:13:39.608360 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.608303 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e05343-e0bb-4813-bebf-f2319a90329d" containerName="manager" Jul 09 14:13:39.608396 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.608382 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e05343-e0bb-4813-bebf-f2319a90329d" containerName="manager" Jul 09 14:13:39.613177 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.613153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.616561 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.616533 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 14:13:39.625166 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.625136 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l"] Jul 09 14:13:39.702308 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.702266 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf420269-9854-43f5-86b5-6f3532e82fe5-openshift-service-ca\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.702501 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.702342 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.702501 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.702449 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7wk\" (UniqueName: \"kubernetes.io/projected/bf420269-9854-43f5-86b5-6f3532e82fe5-kube-api-access-qh7wk\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.804043 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.804001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf420269-9854-43f5-86b5-6f3532e82fe5-openshift-service-ca\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.804300 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.804056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.804300 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.804106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7wk\" (UniqueName: \"kubernetes.io/projected/bf420269-9854-43f5-86b5-6f3532e82fe5-kube-api-access-qh7wk\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.804300 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:13:39.804211 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 09 14:13:39.804506 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:13:39.804350 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls podName:bf420269-9854-43f5-86b5-6f3532e82fe5 nodeName:}" failed. No retries permitted until 2026-07-09 14:13:40.304328977 +0000 UTC m=+681.943003728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls") pod "maas-api-5455645bcf-zjv7l" (UID: "bf420269-9854-43f5-86b5-6f3532e82fe5") : secret "maas-api-serving-cert" not found Jul 09 14:13:39.804800 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.804776 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf420269-9854-43f5-86b5-6f3532e82fe5-openshift-service-ca\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:39.813991 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:39.813960 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7wk\" (UniqueName: \"kubernetes.io/projected/bf420269-9854-43f5-86b5-6f3532e82fe5-kube-api-access-qh7wk\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:40.158213 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.158174 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-62vvg"] Jul 09 14:13:40.162439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.162413 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.165607 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.165580 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 14:13:40.165607 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.165599 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7qlk5\"" Jul 09 14:13:40.176584 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.176555 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-62vvg"] Jul 09 14:13:40.190837 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.190806 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zcg89"] Jul 09 14:13:40.194631 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.194602 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.202551 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.202509 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zcg89"] Jul 09 14:13:40.308906 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.308863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:40.309108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.308926 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d93ba7a-e305-4286-82d7-d2cc188c6e14-plugins-config-volume\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.309108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.308963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdgpm\" (UniqueName: \"kubernetes.io/projected/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-kube-api-access-xdgpm\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.309275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.309106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-plugins-config-volume\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.309275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.309155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knvxl\" (UniqueName: \"kubernetes.io/projected/9d93ba7a-e305-4286-82d7-d2cc188c6e14-kube-api-access-knvxl\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.311615 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.311587 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf420269-9854-43f5-86b5-6f3532e82fe5-maas-api-tls\") pod \"maas-api-5455645bcf-zjv7l\" (UID: \"bf420269-9854-43f5-86b5-6f3532e82fe5\") " pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:40.410405 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.410328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-plugins-config-volume\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.410405 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.410368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knvxl\" (UniqueName: \"kubernetes.io/projected/9d93ba7a-e305-4286-82d7-d2cc188c6e14-kube-api-access-knvxl\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.410613 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.410440 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d93ba7a-e305-4286-82d7-d2cc188c6e14-plugins-config-volume\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.410613 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.410462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdgpm\" (UniqueName: \"kubernetes.io/projected/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-kube-api-access-xdgpm\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.411135 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.411112 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-plugins-config-volume\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.411175 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.411115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d93ba7a-e305-4286-82d7-d2cc188c6e14-plugins-config-volume\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.419190 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.419142 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdgpm\" (UniqueName: \"kubernetes.io/projected/f1ce9e10-f264-4251-ae7b-bab6d3ecf289-kube-api-access-xdgpm\") pod \"payload-pre-processing-7964457696-62vvg\" (UID: \"f1ce9e10-f264-4251-ae7b-bab6d3ecf289\") " pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.419394 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.419198 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knvxl\" (UniqueName: \"kubernetes.io/projected/9d93ba7a-e305-4286-82d7-d2cc188c6e14-kube-api-access-knvxl\") pod \"payload-processing-58f86c7798-zcg89\" (UID: \"9d93ba7a-e305-4286-82d7-d2cc188c6e14\") " pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.476133 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.476092 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:40.507403 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.507359 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:40.525439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.525409 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:40.676286 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.676255 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-62vvg"] Jul 09 14:13:40.678129 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:13:40.678093 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ce9e10_f264_4251_ae7b_bab6d3ecf289.slice/crio-70fac062548d92d0d1cae622809838c7722437b130ca5b2e4f70a97d956f2bac WatchSource:0}: Error finding container 70fac062548d92d0d1cae622809838c7722437b130ca5b2e4f70a97d956f2bac: Status 404 returned error can't find the container with id 70fac062548d92d0d1cae622809838c7722437b130ca5b2e4f70a97d956f2bac Jul 09 14:13:40.705178 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.705147 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zcg89"] Jul 09 14:13:40.706689 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:13:40.706662 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d93ba7a_e305_4286_82d7_d2cc188c6e14.slice/crio-4ec284787ce2ba001d93e9b99e74f0f86c63cee8579c2f8620071d28361d3471 WatchSource:0}: Error finding container 4ec284787ce2ba001d93e9b99e74f0f86c63cee8579c2f8620071d28361d3471: Status 404 returned error can't find the container with id 4ec284787ce2ba001d93e9b99e74f0f86c63cee8579c2f8620071d28361d3471 Jul 09 14:13:40.721513 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:40.721483 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l"] Jul 09 14:13:41.697011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:41.696952 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" event={"ID":"bf420269-9854-43f5-86b5-6f3532e82fe5","Type":"ContainerStarted","Data":"7cc78dbe0cbf22984f0a2484dc67caa058346beab1c383b88f5c06a2e29cf875"} Jul 09 14:13:41.698637 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:41.698601 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" event={"ID":"f1ce9e10-f264-4251-ae7b-bab6d3ecf289","Type":"ContainerStarted","Data":"70fac062548d92d0d1cae622809838c7722437b130ca5b2e4f70a97d956f2bac"} Jul 09 14:13:41.700375 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:41.700335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" event={"ID":"9d93ba7a-e305-4286-82d7-d2cc188c6e14","Type":"ContainerStarted","Data":"4ec284787ce2ba001d93e9b99e74f0f86c63cee8579c2f8620071d28361d3471"} Jul 09 14:13:43.717709 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.717661 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" event={"ID":"bf420269-9854-43f5-86b5-6f3532e82fe5","Type":"ContainerStarted","Data":"845576ea8b24fd6858be18d625efe82989c694cd28d0167bfc982bb3d8647d92"} Jul 09 14:13:43.718206 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.717762 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:43.719741 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.719713 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" event={"ID":"f1ce9e10-f264-4251-ae7b-bab6d3ecf289","Type":"ContainerStarted","Data":"398a516b50a1f90dd1673de2d8c24f863bbae8c5e02b6410caacf669ec52c9af"} Jul 09 14:13:43.719882 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.719804 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:13:43.721770 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.721744 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" event={"ID":"9d93ba7a-e305-4286-82d7-d2cc188c6e14","Type":"ContainerStarted","Data":"042251c1b9cdbd43ba5fde048049e96b22aaabe8a4aae5bda9aee23732c20037"} Jul 09 14:13:43.721900 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.721879 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:43.737201 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.737148 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" podStartSLOduration=2.14902303 podStartE2EDuration="4.737132021s" totalImagesPullingTime="2.588108991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:13:43.735131327 +0000 UTC m=+685.373806134" watchObservedRunningTime="2026-07-09 14:13:43.737132021 +0000 UTC m=+685.375806794" Jul 09 14:13:43.754154 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:43.754103 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" podStartSLOduration=1.22719661 podStartE2EDuration="3.754086893s" totalImagesPullingTime="2.526890283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:13:43.752533721 +0000 UTC m=+685.391208558" watchObservedRunningTime="2026-07-09 14:13:43.754086893 +0000 UTC m=+685.392761666" Jul 09 14:13:48.761872 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.761836 2576 generic.go:356] "Generic (PLEG): container finished" podID="d863e139-fe8f-44cd-b968-cf8c45a74d57" containerID="7e4dceec9c562e9d755ca4bca534d68a9f96d692567c53ac1de2baae1eac1d6c" exitCode=0 Jul 09 14:13:48.762349 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.761911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" event={"ID":"d863e139-fe8f-44cd-b968-cf8c45a74d57","Type":"ContainerDied","Data":"7e4dceec9c562e9d755ca4bca534d68a9f96d692567c53ac1de2baae1eac1d6c"} Jul 09 14:13:48.762402 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.762353 2576 scope.go:117] "RemoveContainer" containerID="7e4dceec9c562e9d755ca4bca534d68a9f96d692567c53ac1de2baae1eac1d6c" Jul 09 14:13:48.763384 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.763352 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:13:48.763445 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.763427 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:13:48.763482 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.763458 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 14:13:48.787000 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:48.786954 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" podStartSLOduration=6.228349483 podStartE2EDuration="8.786938829s" totalImagesPullingTime="2.558589346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:13:43.774777794 +0000 UTC m=+685.413452571" watchObservedRunningTime="2026-07-09 14:13:48.786938829 +0000 UTC m=+690.425613604" Jul 09 14:13:49.734600 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:49.734565 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5455645bcf-zjv7l" Jul 09 14:13:49.796253 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:49.793218 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" event={"ID":"d863e139-fe8f-44cd-b968-cf8c45a74d57","Type":"ContainerStarted","Data":"9bd584e53ba6a45047a43afd03d297130ffbc6d998c388123f548e3654d441b1"} Jul 09 14:13:50.613351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:50.613308 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:13:50.613614 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:50.613589 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2rp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:13:50.613673 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:50.613656 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" podUID="d863e139-fe8f-44cd-b968-cf8c45a74d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:13:51.613751 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:51.613710 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2rp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:13:51.614148 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:51.613768 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" podUID="d863e139-fe8f-44cd-b968-cf8c45a74d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:13:52.613614 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:52.613578 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2rp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:13:52.613804 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:52.613635 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" podUID="d863e139-fe8f-44cd-b968-cf8c45a74d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:13:53.613909 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:53.613865 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2rp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:13:53.614358 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:53.613939 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" podUID="d863e139-fe8f-44cd-b968-cf8c45a74d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:13:54.617789 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:54.617755 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:13:54.618169 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:54.618035 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:13:54.618702 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:54.618683 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2rp67" Jul 09 14:13:54.730287 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:54.730203 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-zcg89" Jul 09 14:13:54.730490 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:13:54.730384 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-62vvg" Jul 09 14:14:01.619301 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.619262 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t"] Jul 09 14:14:01.624417 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.624391 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.627178 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.627152 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 14:14:01.628519 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.628490 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 14:14:01.628519 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.628508 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 14:14:01.628744 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.628519 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f6xmw\"" Jul 09 14:14:01.634320 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.634295 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t"] Jul 09 14:14:01.726181 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726138 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk2vg\" (UniqueName: \"kubernetes.io/projected/14397047-3272-413d-bbb8-eee19da88a78-kube-api-access-pk2vg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.726380 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.726380 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.726380 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726305 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14397047-3272-413d-bbb8-eee19da88a78-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.726380 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726327 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.726521 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.726422 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827545 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk2vg\" (UniqueName: \"kubernetes.io/projected/14397047-3272-413d-bbb8-eee19da88a78-kube-api-access-pk2vg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827696 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14397047-3272-413d-bbb8-eee19da88a78-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.827738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.828005 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.827967 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.828065 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.828031 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.828144 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.828121 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.830066 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.830040 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14397047-3272-413d-bbb8-eee19da88a78-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.830330 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.830308 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14397047-3272-413d-bbb8-eee19da88a78-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.836052 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.836024 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk2vg\" (UniqueName: \"kubernetes.io/projected/14397047-3272-413d-bbb8-eee19da88a78-kube-api-access-pk2vg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t\" (UID: \"14397047-3272-413d-bbb8-eee19da88a78\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:01.937047 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:01.936931 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:02.087104 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:02.087052 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t"] Jul 09 14:14:02.088531 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:14:02.088499 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14397047_3272_413d_bbb8_eee19da88a78.slice/crio-a9ff2fc2c3df8c9de5f7100bebee03253bd215f5ac70126b716150ba04eef438 WatchSource:0}: Error finding container a9ff2fc2c3df8c9de5f7100bebee03253bd215f5ac70126b716150ba04eef438: Status 404 returned error can't find the container with id a9ff2fc2c3df8c9de5f7100bebee03253bd215f5ac70126b716150ba04eef438 Jul 09 14:14:02.877387 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:02.877352 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" event={"ID":"14397047-3272-413d-bbb8-eee19da88a78","Type":"ContainerStarted","Data":"a9ff2fc2c3df8c9de5f7100bebee03253bd215f5ac70126b716150ba04eef438"} Jul 09 14:14:03.884918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:03.884877 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" event={"ID":"14397047-3272-413d-bbb8-eee19da88a78","Type":"ContainerStarted","Data":"61d23e51719817eaa5ffaf487d365171e2dcfbde48a8af5c074a575a98968e23"} Jul 09 14:14:03.905808 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:03.905752 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" podStartSLOduration=1.658959195 podStartE2EDuration="2.905733959s" totalImagesPullingTime="1.246774764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:14:03.902664328 +0000 UTC m=+705.541339102" watchObservedRunningTime="2026-07-09 14:14:03.905733959 +0000 UTC m=+705.544408734" Jul 09 14:14:11.937919 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:11.937833 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:11.938297 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:11.937913 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:11.950706 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:11.950676 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:12.950265 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:12.950208 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t" Jul 09 14:14:21.516460 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.516418 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc"] Jul 09 14:14:21.522076 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.522046 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.525567 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.525535 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 14:14:21.533983 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.533954 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc"] Jul 09 14:14:21.629502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629458 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65lxn\" (UniqueName: \"kubernetes.io/projected/a20b32b4-8c0e-41d9-979d-5da99189d743-kube-api-access-65lxn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.629679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20b32b4-8c0e-41d9-979d-5da99189d743-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.629679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.629679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.629679 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629645 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.629820 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.629690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730708 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65lxn\" (UniqueName: \"kubernetes.io/projected/a20b32b4-8c0e-41d9-979d-5da99189d743-kube-api-access-65lxn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20b32b4-8c0e-41d9-979d-5da99189d743-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.730903 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.730813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.731312 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.731283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.731400 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.731314 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.731400 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.731346 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.733105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.733075 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20b32b4-8c0e-41d9-979d-5da99189d743-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.733421 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.733403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20b32b4-8c0e-41d9-979d-5da99189d743-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.740753 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.740727 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65lxn\" (UniqueName: \"kubernetes.io/projected/a20b32b4-8c0e-41d9-979d-5da99189d743-kube-api-access-65lxn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpjc\" (UID: \"a20b32b4-8c0e-41d9-979d-5da99189d743\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.836415 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.836378 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:21.993279 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:21.993243 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc"] Jul 09 14:14:21.994518 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:14:21.994491 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20b32b4_8c0e_41d9_979d_5da99189d743.slice/crio-9bc8fbcfe6015b0806cf4166307e90d50868b6e4c6e3519b68c1e5abf99b297f WatchSource:0}: Error finding container 9bc8fbcfe6015b0806cf4166307e90d50868b6e4c6e3519b68c1e5abf99b297f: Status 404 returned error can't find the container with id 9bc8fbcfe6015b0806cf4166307e90d50868b6e4c6e3519b68c1e5abf99b297f Jul 09 14:14:22.003179 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:22.003151 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" event={"ID":"a20b32b4-8c0e-41d9-979d-5da99189d743","Type":"ContainerStarted","Data":"9bc8fbcfe6015b0806cf4166307e90d50868b6e4c6e3519b68c1e5abf99b297f"} Jul 09 14:14:23.011110 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:23.011072 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" event={"ID":"a20b32b4-8c0e-41d9-979d-5da99189d743","Type":"ContainerStarted","Data":"b6cf4800fb1d68cc7190cf6dfd61c2d33096041889e645ad6258026e05c7cff6"} Jul 09 14:14:23.032286 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:23.032213 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" podStartSLOduration=1.802765274 podStartE2EDuration="2.03219682s" totalImagesPullingTime="229.431546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:14:23.030144908 +0000 UTC m=+724.668819703" watchObservedRunningTime="2026-07-09 14:14:23.03219682 +0000 UTC m=+724.670871593" Jul 09 14:14:31.837296 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:31.837254 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:31.837702 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:31.837344 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:31.850199 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:31.850163 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:32.078871 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:32.078835 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpjc" Jul 09 14:14:36.425808 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.425771 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh"] Jul 09 14:14:36.433122 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.433090 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.436840 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.436808 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 14:14:36.447834 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.447755 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh"] Jul 09 14:14:36.473667 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.473620 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.473976 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.473954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.474266 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.474174 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbl9f\" (UniqueName: \"kubernetes.io/projected/617d3e1a-426d-4366-b410-0cb0da8aad4b-kube-api-access-cbl9f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.474385 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.474263 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/617d3e1a-426d-4366-b410-0cb0da8aad4b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.474385 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.474373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.474504 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.474398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.575988 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.575948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.575988 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.575995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576105 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576130 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbl9f\" (UniqueName: \"kubernetes.io/projected/617d3e1a-426d-4366-b410-0cb0da8aad4b-kube-api-access-cbl9f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576275 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576171 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/617d3e1a-426d-4366-b410-0cb0da8aad4b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576455 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576500 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.576769 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.576740 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.578656 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.578629 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/617d3e1a-426d-4366-b410-0cb0da8aad4b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.579272 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.579251 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/617d3e1a-426d-4366-b410-0cb0da8aad4b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.584708 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.584674 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbl9f\" (UniqueName: \"kubernetes.io/projected/617d3e1a-426d-4366-b410-0cb0da8aad4b-kube-api-access-cbl9f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh\" (UID: \"617d3e1a-426d-4366-b410-0cb0da8aad4b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.779597 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.779499 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:36.933107 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:36.933074 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh"] Jul 09 14:14:36.934252 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:14:36.934214 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod617d3e1a_426d_4366_b410_0cb0da8aad4b.slice/crio-51c78a16e8a64c3ea0b30e7d0a6e2fe302d56a4787e552e5c895a808df5791e8 WatchSource:0}: Error finding container 51c78a16e8a64c3ea0b30e7d0a6e2fe302d56a4787e552e5c895a808df5791e8: Status 404 returned error can't find the container with id 51c78a16e8a64c3ea0b30e7d0a6e2fe302d56a4787e552e5c895a808df5791e8 Jul 09 14:14:37.100132 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:37.100098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" event={"ID":"617d3e1a-426d-4366-b410-0cb0da8aad4b","Type":"ContainerStarted","Data":"51c78a16e8a64c3ea0b30e7d0a6e2fe302d56a4787e552e5c895a808df5791e8"} Jul 09 14:14:38.108325 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:38.108288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" event={"ID":"617d3e1a-426d-4366-b410-0cb0da8aad4b","Type":"ContainerStarted","Data":"d5c6786c200d7b77c003fd706b377174a0a03102268288520720d07e72a7040c"} Jul 09 14:14:38.128124 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:38.128052 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" podStartSLOduration=1.9236016230000001 podStartE2EDuration="2.128036729s" totalImagesPullingTime="204.435106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:14:38.126683138 +0000 UTC m=+739.765357932" watchObservedRunningTime="2026-07-09 14:14:38.128036729 +0000 UTC m=+739.766711502" Jul 09 14:14:46.780644 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:46.780596 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:46.781074 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:46.780668 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:46.793265 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:46.793217 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:14:47.175691 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:14:47.175660 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh" Jul 09 14:15:21.823922 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.823878 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b87747b57-ncddr"] Jul 09 14:15:21.827664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.827639 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:21.832079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.832052 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xm6t4\"" Jul 09 14:15:21.832079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.832067 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 14:15:21.836262 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.836218 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b87747b57-ncddr"] Jul 09 14:15:21.924848 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.924804 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvbw9\" (UniqueName: \"kubernetes.io/projected/c71e360b-317d-482e-a822-fb5e1d054663-kube-api-access-zvbw9\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:21.925160 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:21.925141 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c71e360b-317d-482e-a822-fb5e1d054663-tls-cert\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.025998 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.025960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c71e360b-317d-482e-a822-fb5e1d054663-tls-cert\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.026164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.026021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvbw9\" (UniqueName: \"kubernetes.io/projected/c71e360b-317d-482e-a822-fb5e1d054663-kube-api-access-zvbw9\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.028592 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.028557 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c71e360b-317d-482e-a822-fb5e1d054663-tls-cert\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.034150 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.034125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvbw9\" (UniqueName: \"kubernetes.io/projected/c71e360b-317d-482e-a822-fb5e1d054663-kube-api-access-zvbw9\") pod \"authorino-b87747b57-ncddr\" (UID: \"c71e360b-317d-482e-a822-fb5e1d054663\") " pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.139298 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.139180 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b87747b57-ncddr" Jul 09 14:15:22.299374 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.299332 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b87747b57-ncddr"] Jul 09 14:15:22.302763 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:15:22.302729 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc71e360b_317d_482e_a822_fb5e1d054663.slice/crio-2167764abb5bde627fee271c21943fae1187c845e1e54988186b8ee87b084c1e WatchSource:0}: Error finding container 2167764abb5bde627fee271c21943fae1187c845e1e54988186b8ee87b084c1e: Status 404 returned error can't find the container with id 2167764abb5bde627fee271c21943fae1187c845e1e54988186b8ee87b084c1e Jul 09 14:15:22.399343 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:22.399254 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b87747b57-ncddr" event={"ID":"c71e360b-317d-482e-a822-fb5e1d054663","Type":"ContainerStarted","Data":"2167764abb5bde627fee271c21943fae1187c845e1e54988186b8ee87b084c1e"} Jul 09 14:15:25.420851 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:25.420811 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b87747b57-ncddr" event={"ID":"c71e360b-317d-482e-a822-fb5e1d054663","Type":"ContainerStarted","Data":"8f91f282584e3bd51b0eb4d10c36c3884504ee32d09f21e779fea3c86c2fbbc8"} Jul 09 14:15:25.442472 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:15:25.442415 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b87747b57-ncddr" podStartSLOduration=2.040262578 podStartE2EDuration="4.442396345s" totalImagesPullingTime="2.402133767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:25.441090888 +0000 UTC m=+787.079765665" watchObservedRunningTime="2026-07-09 14:15:25.442396345 +0000 UTC m=+787.081071118" Jul 09 14:17:02.095673 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.095630 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:17:02.096150 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.095871 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" podUID="c537bea8-e746-41cd-b573-3e4f98cefd67" containerName="manager" containerID="cri-o://f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc" gracePeriod=30 Jul 09 14:17:02.365562 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.365532 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:17:02.464311 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.464270 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg5tf\" (UniqueName: \"kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf\") pod \"c537bea8-e746-41cd-b573-3e4f98cefd67\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " Jul 09 14:17:02.464513 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.464432 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert\") pod \"c537bea8-e746-41cd-b573-3e4f98cefd67\" (UID: \"c537bea8-e746-41cd-b573-3e4f98cefd67\") " Jul 09 14:17:02.466506 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.466474 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c537bea8-e746-41cd-b573-3e4f98cefd67" (UID: "c537bea8-e746-41cd-b573-3e4f98cefd67"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:17:02.466627 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.466474 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf" (OuterVolumeSpecName: "kube-api-access-xg5tf") pod "c537bea8-e746-41cd-b573-3e4f98cefd67" (UID: "c537bea8-e746-41cd-b573-3e4f98cefd67"). InnerVolumeSpecName "kube-api-access-xg5tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:17:02.565818 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.565772 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c537bea8-e746-41cd-b573-3e4f98cefd67-webhook-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:17:02.565818 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:02.565809 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg5tf\" (UniqueName: \"kubernetes.io/projected/c537bea8-e746-41cd-b573-3e4f98cefd67-kube-api-access-xg5tf\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\"" Jul 09 14:17:03.043138 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.043099 2576 generic.go:356] "Generic (PLEG): container finished" podID="c537bea8-e746-41cd-b573-3e4f98cefd67" containerID="f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc" exitCode=0 Jul 09 14:17:03.043359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.043168 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" Jul 09 14:17:03.043359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.043183 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" event={"ID":"c537bea8-e746-41cd-b573-3e4f98cefd67","Type":"ContainerDied","Data":"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc"} Jul 09 14:17:03.043359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.043279 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dbd89dd4-f9wgx" event={"ID":"c537bea8-e746-41cd-b573-3e4f98cefd67","Type":"ContainerDied","Data":"185207c91a51c7dc43303ef0b83bcd56fe956446cd6448309739c4794c14f70b"} Jul 09 14:17:03.043359 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.043303 2576 scope.go:117] "RemoveContainer" containerID="f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc" Jul 09 14:17:03.053164 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.053147 2576 scope.go:117] "RemoveContainer" containerID="f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc" Jul 09 14:17:03.053457 ip-10-0-140-134 kubenswrapper[2576]: E0709 14:17:03.053434 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc\": container with ID starting with f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc not found: ID does not exist" containerID="f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc" Jul 09 14:17:03.053567 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.053467 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc"} err="failed to get container status \"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc\": rpc error: code = NotFound desc = could not find container \"f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc\": container with ID starting with f85e1d669886b4a2d21cf5e50aeae45e6023b5022a223e35a6aeef004c45b9cc not found: ID does not exist" Jul 09 14:17:03.062151 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.062118 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:17:03.065288 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:03.065263 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65dbd89dd4-f9wgx"] Jul 09 14:17:04.896796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:04.896758 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c537bea8-e746-41cd-b573-3e4f98cefd67" path="/var/lib/kubelet/pods/c537bea8-e746-41cd-b573-3e4f98cefd67/volumes" Jul 09 14:17:18.886889 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:18.886855 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:17:18.889818 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:18.889789 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:17:18.891177 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:18.891156 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:17:18.894448 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:17:18.894429 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:22:18.937015 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:22:18.936987 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:22:18.939252 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:22:18.939207 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:22:18.940953 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:22:18.940933 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:22:18.943085 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:22:18.943063 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:27:18.982890 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:27:18.982862 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:27:18.985479 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:27:18.984875 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:27:18.986578 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:27:18.986560 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:27:18.988596 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:27:18.988579 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:32:19.029045 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:32:19.028922 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:32:19.035055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:32:19.030707 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:32:19.035055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:32:19.032549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:32:19.035055 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:32:19.034575 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:37:19.073502 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:37:19.073470 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:37:19.075900 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:37:19.075883 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:37:19.076951 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:37:19.076928 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:37:19.079131 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:37:19.079108 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:42:19.117747 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:42:19.117632 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:42:19.121749 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:42:19.121350 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:42:19.121861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:42:19.121790 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:42:19.125132 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:42:19.125111 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:47:19.176823 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:19.176709 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:47:19.181168 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:19.181142 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:47:19.181989 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:19.181952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:47:19.186174 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:19.186148 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lmg5b_b7ca6750-e97a-4417-bce0-567a03402717/ovn-acl-logging/0.log" Jul 09 14:47:28.359756 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:28.359720 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b87747b57-ncddr_c71e360b-317d-482e-a822-fb5e1d054663/authorino/0.log" Jul 09 14:47:33.228468 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:33.228436 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4ckzr_f428dd9b-c596-4ff0-8061-721507aca043/manager/0.log" Jul 09 14:47:33.354662 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:33.354629 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q8q42_8dc5e1be-05a1-4a77-9a72-737df6ac5475/manager/0.log" Jul 09 14:47:33.599338 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:33.599307 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cndlx_4cfae5e3-218f-41e7-9c62-258cabbb800e/server/0.log" Jul 09 14:47:33.716073 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:33.716037 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tfxx6_3c70fa9f-1b94-4e60-aec5-3045950aa46d/manager/0.log" Jul 09 14:47:34.096136 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:34.096104 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f67568454-wtfsp_b7af6e15-9a7d-4ba3-868a-d64570b3ffff/copy-manifests/0.log" Jul 09 14:47:34.109057 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:34.109029 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f67568454-wtfsp_b7af6e15-9a7d-4ba3-868a-d64570b3ffff/manager/0.log" Jul 09 14:47:34.492335 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:34.492250 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5455645bcf-zjv7l_bf420269-9854-43f5-86b5-6f3532e82fe5/maas-api/0.log" Jul 09 14:47:34.966474 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:34.966445 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6pgfw_b684a681-7cf3-4cca-94e3-627306e2f57a/postgres/0.log" Jul 09 14:47:35.788256 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.788208 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/util/0.log" Jul 09 14:47:35.794377 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.794354 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/pull/0.log" Jul 09 14:47:35.800076 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.800052 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/extract/0.log" Jul 09 14:47:35.922738 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.922706 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/util/0.log" Jul 09 14:47:35.930896 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.930865 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/pull/0.log" Jul 09 14:47:35.938748 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:35.938724 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/extract/0.log" Jul 09 14:47:36.052616 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.052533 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/util/0.log" Jul 09 14:47:36.059347 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.059315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/pull/0.log" Jul 09 14:47:36.068865 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.068843 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/extract/0.log" Jul 09 14:47:36.180166 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.180133 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/extract/0.log" Jul 09 14:47:36.186105 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.186084 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/util/0.log" Jul 09 14:47:36.192191 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.192172 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/pull/0.log" Jul 09 14:47:36.313870 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.313829 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b87747b57-ncddr_c71e360b-317d-482e-a822-fb5e1d054663/authorino/0.log" Jul 09 14:47:36.429960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.429925 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ss4nw_2d8ffc9c-25d5-419c-9433-39d3fa0eea27/manager/0.log" Jul 09 14:47:36.542407 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.542376 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t9hzx_952b6856-5488-4439-8b8f-cd2d0cf63b4c/manager/0.log" Jul 09 14:47:36.655124 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.655041 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-trwbx_6fc54ada-51d6-47dc-a8b5-5c474a7399e9/kuadrant-console-plugin/0.log" Jul 09 14:47:36.776413 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:36.776380 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jgv9q_fbe80fa5-49b9-4941-844f-44293610551c/registry-server/0.log" Jul 09 14:47:37.211021 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:37.210984 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bnqcw_eb315145-8661-44a4-921d-be7a510b6b82/manager/0.log" Jul 09 14:47:37.908161 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:37.908128 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6dccbd4-wkmbf_7eb16642-3fdf-4ffe-bdab-76d0510a93ef/kube-auth-proxy/0.log" Jul 09 14:47:38.025513 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.025483 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/1.log" Jul 09 14:47:38.147398 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.147367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-62vvg_f1ce9e10-f264-4251-ae7b-bab6d3ecf289/payload-pre-processing/0.log" Jul 09 14:47:38.260796 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.260698 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zcg89_9d93ba7a-e305-4286-82d7-d2cc188c6e14/payload-processing/0.log" Jul 09 14:47:38.375079 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.375050 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bb56f78d-q42xs_0dece533-f971-48c8-972c-c312fe875473/router/0.log" Jul 09 14:47:38.716483 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.716446 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-g5q9t_14397047-3272-413d-bbb8-eee19da88a78/main/0.log" Jul 09 14:47:38.940085 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:38.940056 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rrpjc_a20b32b4-8c0e-41d9-979d-5da99189d743/main/0.log" Jul 09 14:47:39.292439 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:39.292408 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-jsjvh_617d3e1a-426d-4366-b410-0cb0da8aad4b/main/0.log" Jul 09 14:47:46.726145 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:46.726112 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8q4t5_7ae8c8cd-c765-411a-a418-a2387ac28c01/global-pull-secret-syncer/0.log" Jul 09 14:47:46.878629 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:46.878596 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qhggs_2e27433c-2659-4dc2-9405-b9f3851d935a/konnectivity-agent/0.log" Jul 09 14:47:46.978438 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:46.978357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-134.ec2.internal_f46bed090be62d9ef320eca39a1d183b/haproxy/0.log" Jul 09 14:47:51.665059 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.664971 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/extract/0.log" Jul 09 14:47:51.684667 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.684637 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/util/0.log" Jul 09 14:47:51.704870 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.704837 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594wqgc_8a2b1bc9-58fc-48ab-9e90-342afbb87cfb/pull/0.log" Jul 09 14:47:51.746244 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.746213 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/extract/0.log" Jul 09 14:47:51.774467 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.774418 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/util/0.log" Jul 09 14:47:51.794841 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.794810 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p8kkg_41cf314c-e445-4804-983f-3b1f74649e13/pull/0.log" Jul 09 14:47:51.818861 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.818835 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/extract/0.log" Jul 09 14:47:51.843083 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.843056 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/util/0.log" Jul 09 14:47:51.864386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.864357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8mpr_64f99786-912b-4767-9be1-c79fa08db0a7/pull/0.log" Jul 09 14:47:51.893735 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.893706 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/extract/0.log" Jul 09 14:47:51.914331 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.914307 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/util/0.log" Jul 09 14:47:51.940107 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:51.940020 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cztp_f1494dff-e2cc-40cd-a938-2ba2d420f41d/pull/0.log" Jul 09 14:47:52.177905 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.177874 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b87747b57-ncddr_c71e360b-317d-482e-a822-fb5e1d054663/authorino/0.log" Jul 09 14:47:52.203146 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.203072 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ss4nw_2d8ffc9c-25d5-419c-9433-39d3fa0eea27/manager/0.log" Jul 09 14:47:52.228280 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.228250 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t9hzx_952b6856-5488-4439-8b8f-cd2d0cf63b4c/manager/0.log" Jul 09 14:47:52.251177 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.251151 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-trwbx_6fc54ada-51d6-47dc-a8b5-5c474a7399e9/kuadrant-console-plugin/0.log" Jul 09 14:47:52.288799 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.288770 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jgv9q_fbe80fa5-49b9-4941-844f-44293610551c/registry-server/0.log" Jul 09 14:47:52.548543 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:52.548452 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bnqcw_eb315145-8661-44a4-921d-be7a510b6b82/manager/0.log" Jul 09 14:47:54.385918 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.385883 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/alertmanager/0.log" Jul 09 14:47:54.407081 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.407052 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/config-reloader/0.log" Jul 09 14:47:54.428632 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.428576 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/kube-rbac-proxy-web/0.log" Jul 09 14:47:54.450351 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.450320 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/kube-rbac-proxy/0.log" Jul 09 14:47:54.472859 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.472810 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/kube-rbac-proxy-metric/0.log" Jul 09 14:47:54.496514 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.496483 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/prom-label-proxy/0.log" Jul 09 14:47:54.519011 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.518984 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d022a56d-d3ab-482f-b1a5-491647894c01/init-config-reloader/0.log" Jul 09 14:47:54.560815 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.560782 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-nwltl_205237ba-20da-4c38-8008-9a4403f17b1f/cluster-monitoring-operator/0.log" Jul 09 14:47:54.586240 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.586197 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-state-metrics/0.log" Jul 09 14:47:54.603623 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.603599 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-rbac-proxy-main/0.log" Jul 09 14:47:54.624363 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.624337 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9sc26_0deb68b5-ebd8-42f1-81b8-a9422077801d/kube-rbac-proxy-self/0.log" Jul 09 14:47:54.653049 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.652976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-798bd77ff7-wfg7b_62739d4a-d3fa-41d5-a923-1203f4e877e0/metrics-server/0.log" Jul 09 14:47:54.681642 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.681608 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-7r27g_dcfd7a6b-0c44-4d11-8eb8-6f5d7ebfd209/monitoring-plugin/0.log" Jul 09 14:47:54.787603 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.787575 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/node-exporter/0.log" Jul 09 14:47:54.815137 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.815112 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/kube-rbac-proxy/0.log" Jul 09 14:47:54.836718 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:54.836688 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x6zvx_a17465a6-b481-4031-aa11-42699ff72664/init-textfile/0.log" Jul 09 14:47:55.269680 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.269600 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5778d8b68f-9bqch_b4023edb-d68c-458a-8fb1-fb7588f3c6b9/telemeter-client/0.log" Jul 09 14:47:55.289769 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.289706 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5778d8b68f-9bqch_b4023edb-d68c-458a-8fb1-fb7588f3c6b9/reload/0.log" Jul 09 14:47:55.310565 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.310540 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5778d8b68f-9bqch_b4023edb-d68c-458a-8fb1-fb7588f3c6b9/kube-rbac-proxy/0.log" Jul 09 14:47:55.339141 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.339112 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/thanos-query/0.log" Jul 09 14:47:55.360030 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.360008 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-web/0.log" Jul 09 14:47:55.381128 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.381101 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy/0.log" Jul 09 14:47:55.400639 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.400614 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/prom-label-proxy/0.log" Jul 09 14:47:55.419772 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.419749 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-rules/0.log" Jul 09 14:47:55.439909 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:55.439884 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b765cb6-4nsxc_f25adc8f-59d7-4068-93f0-b3278c25e692/kube-rbac-proxy-metrics/0.log" Jul 09 14:47:56.643880 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.643844 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv"] Jul 09 14:47:56.644430 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.644412 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c537bea8-e746-41cd-b573-3e4f98cefd67" containerName="manager" Jul 09 14:47:56.644488 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.644433 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c537bea8-e746-41cd-b573-3e4f98cefd67" containerName="manager" Jul 09 14:47:56.644546 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.644536 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c537bea8-e746-41cd-b573-3e4f98cefd67" containerName="manager" Jul 09 14:47:56.648135 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.648111 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.651027 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.650986 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqx2q\"/\"openshift-service-ca.crt\"" Jul 09 14:47:56.651161 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.651060 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqx2q\"/\"kube-root-ca.crt\"" Jul 09 14:47:56.652376 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.652354 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gqx2q\"/\"default-dockercfg-4vx79\"" Jul 09 14:47:56.660244 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.660208 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-pw5d6_2b62edca-81e3-4de9-88e0-bd08cd290874/networking-console-plugin/0.log" Jul 09 14:47:56.661068 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.661047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv"] Jul 09 14:47:56.784094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.784051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-lib-modules\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.784094 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.784094 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-podres\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.784386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.784122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwg8n\" (UniqueName: \"kubernetes.io/projected/26dc882e-9ba0-4e89-b66a-a18af244385d-kube-api-access-fwg8n\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.784386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.784212 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-proc\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.784386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.784301 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-sys\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885664 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885622 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-sys\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-lib-modules\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-sys\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-podres\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885793 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwg8n\" (UniqueName: \"kubernetes.io/projected/26dc882e-9ba0-4e89-b66a-a18af244385d-kube-api-access-fwg8n\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885833 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-podres\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.885853 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-proc\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.886109 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-proc\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.886109 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.885947 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26dc882e-9ba0-4e89-b66a-a18af244385d-lib-modules\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.894306 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.894209 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwg8n\" (UniqueName: \"kubernetes.io/projected/26dc882e-9ba0-4e89-b66a-a18af244385d-kube-api-access-fwg8n\") pod \"perf-node-gather-daemonset-zfncv\" (UID: \"26dc882e-9ba0-4e89-b66a-a18af244385d\") " pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:56.959586 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:56.959552 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:57.109569 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.109530 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv"] Jul 09 14:47:57.110976 ip-10-0-140-134 kubenswrapper[2576]: W0709 14:47:57.110947 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod26dc882e_9ba0_4e89_b66a_a18af244385d.slice/crio-e231e06dfe61641a4c8efc4749ba3a7f2539cfc01f6110040334e070e3afad30 WatchSource:0}: Error finding container e231e06dfe61641a4c8efc4749ba3a7f2539cfc01f6110040334e070e3afad30: Status 404 returned error can't find the container with id e231e06dfe61641a4c8efc4749ba3a7f2539cfc01f6110040334e070e3afad30 Jul 09 14:47:57.361986 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.361957 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/2.log" Jul 09 14:47:57.370323 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.370289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jzqf9_84261dd3-3238-4f0b-b2bc-c453c9963162/console-operator/3.log" Jul 09 14:47:57.891498 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.891460 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b9db479fb-wc9jr_79eee603-324d-49c0-8c47-ac1a2e94dc1b/console/0.log" Jul 09 14:47:57.971012 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.970976 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" event={"ID":"26dc882e-9ba0-4e89-b66a-a18af244385d","Type":"ContainerStarted","Data":"13673231ff35bbe334bf92f2165ab024b225bb5db98cbaf72cef76df0d5c7c30"} Jul 09 14:47:57.971188 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.971029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" event={"ID":"26dc882e-9ba0-4e89-b66a-a18af244385d","Type":"ContainerStarted","Data":"e231e06dfe61641a4c8efc4749ba3a7f2539cfc01f6110040334e070e3afad30"} Jul 09 14:47:57.971188 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.971098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:47:57.993386 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:57.993335 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" podStartSLOduration=1.993320209 podStartE2EDuration="1.993320209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:47:57.990337136 +0000 UTC m=+2739.629011912" watchObservedRunningTime="2026-07-09 14:47:57.993320209 +0000 UTC m=+2739.631994983" Jul 09 14:47:58.542510 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:58.542482 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-dspmr_a2e67427-f275-404c-b9bc-e73e99c38d97/volume-data-source-validator/0.log" Jul 09 14:47:59.625557 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:59.625526 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cvrb_717b6137-b92d-46b8-9a6f-167d54a214e8/dns/0.log" Jul 09 14:47:59.646791 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:59.646762 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cvrb_717b6137-b92d-46b8-9a6f-167d54a214e8/kube-rbac-proxy/0.log" Jul 09 14:47:59.808204 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:47:59.808169 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c4r7k_87cb950f-3838-4997-a2a4-d85b4291d420/dns-node-resolver/0.log" Jul 09 14:48:00.472290 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:00.472259 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gncrb_62c48dea-d112-4f60-8d97-5d1be6f76d5d/node-ca/0.log" Jul 09 14:48:01.729225 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.729193 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6dccbd4-wkmbf_7eb16642-3fdf-4ffe-bdab-76d0510a93ef/kube-auth-proxy/0.log" Jul 09 14:48:01.765823 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.765786 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/0.log" Jul 09 14:48:01.791529 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.791491 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/1.log" Jul 09 14:48:01.832617 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.832584 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-62vvg_f1ce9e10-f264-4251-ae7b-bab6d3ecf289/payload-pre-processing/0.log" Jul 09 14:48:01.879792 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.879761 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zcg89_9d93ba7a-e305-4286-82d7-d2cc188c6e14/payload-processing/0.log" Jul 09 14:48:01.901155 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:01.901129 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bb56f78d-q42xs_0dece533-f971-48c8-972c-c312fe875473/router/0.log" Jul 09 14:48:02.628056 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:02.628027 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jczw6_c302bd4d-03dc-48a3-aa09-0e14dfc592d0/serve-healthcheck-canary/0.log" Jul 09 14:48:03.091749 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.091682 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-p9xm2_9243e7ec-aa19-4e2c-adeb-18debfd84b6f/insights-operator/0.log" Jul 09 14:48:03.093345 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.093321 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-p9xm2_9243e7ec-aa19-4e2c-adeb-18debfd84b6f/insights-operator/1.log" Jul 09 14:48:03.254082 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.254052 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gl4lx_4accd7b0-8f25-4292-9483-34a75cf0cc9d/kube-rbac-proxy/0.log" Jul 09 14:48:03.273393 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.273364 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gl4lx_4accd7b0-8f25-4292-9483-34a75cf0cc9d/exporter/0.log" Jul 09 14:48:03.292100 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.292072 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gl4lx_4accd7b0-8f25-4292-9483-34a75cf0cc9d/extractor/0.log" Jul 09 14:48:03.988040 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:03.988009 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gqx2q/perf-node-gather-daemonset-zfncv" Jul 09 14:48:05.499882 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.499850 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4ckzr_f428dd9b-c596-4ff0-8061-721507aca043/manager/0.log" Jul 09 14:48:05.576960 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.576926 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q8q42_8dc5e1be-05a1-4a77-9a72-737df6ac5475/manager/0.log" Jul 09 14:48:05.663555 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.663528 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cndlx_4cfae5e3-218f-41e7-9c62-258cabbb800e/server/0.log" Jul 09 14:48:05.715651 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.715618 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tfxx6_3c70fa9f-1b94-4e60-aec5-3045950aa46d/manager/0.log" Jul 09 14:48:05.894271 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.894221 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f67568454-wtfsp_b7af6e15-9a7d-4ba3-868a-d64570b3ffff/manager/0.log" Jul 09 14:48:05.912807 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:05.912778 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f67568454-wtfsp_b7af6e15-9a7d-4ba3-868a-d64570b3ffff/copy-manifests/0.log" Jul 09 14:48:07.286245 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.286202 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6dccbd4-wkmbf_7eb16642-3fdf-4ffe-bdab-76d0510a93ef/kube-auth-proxy/0.log" Jul 09 14:48:07.309617 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.309581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/0.log" Jul 09 14:48:07.337078 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.337043 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/1.log" Jul 09 14:48:07.365900 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.365869 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-62vvg_f1ce9e10-f264-4251-ae7b-bab6d3ecf289/payload-pre-processing/0.log" Jul 09 14:48:07.400922 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.400892 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zcg89_9d93ba7a-e305-4286-82d7-d2cc188c6e14/payload-processing/0.log" Jul 09 14:48:07.414108 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:07.414079 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bb56f78d-q42xs_0dece533-f971-48c8-972c-c312fe875473/router/0.log" Jul 09 14:48:08.657224 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.657195 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6dccbd4-wkmbf_7eb16642-3fdf-4ffe-bdab-76d0510a93ef/kube-auth-proxy/0.log" Jul 09 14:48:08.677921 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.677885 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/0.log" Jul 09 14:48:08.705102 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.705066 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2rp67_d863e139-fe8f-44cd-b968-cf8c45a74d57/istio-proxy/1.log" Jul 09 14:48:08.732797 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.732712 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-62vvg_f1ce9e10-f264-4251-ae7b-bab6d3ecf289/payload-pre-processing/0.log" Jul 09 14:48:08.774140 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.774102 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zcg89_9d93ba7a-e305-4286-82d7-d2cc188c6e14/payload-processing/0.log" Jul 09 14:48:08.792093 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:08.792066 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bb56f78d-q42xs_0dece533-f971-48c8-972c-c312fe875473/router/0.log" Jul 09 14:48:12.453496 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:12.453462 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-m5j9w_7a47e50e-4829-4881-bea1-bbeebe26be21/kube-storage-version-migrator-operator/1.log" Jul 09 14:48:12.454362 ip-10-0-140-134 kubenswrapper[2576]: I0709 14:48:12.454340 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-m5j9w_7a47e50e-4829-4881-bea1-bbeebe26be21/kube-storage-version-migrator-operator/0.log"